Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Home
  • Home
EnglishFrançais
European Journal of Applied Mathematics European Journal of Applied Mathematics

Article contents

Deep learning models for global coordinate transformations that linearise PDEs

Part of: Approximation methods and numerical treatment of dynamical systems Numerical problems in dynamical systems Partial differential equations Artificial intelligence (68Txx)

Published online by Cambridge University Press:  24 September 2020

CRAIG GIN [Opens in a new window] ,
BETHANY LUSCH [Opens in a new window] ,
STEVEN L. BRUNTON  and
J. NATHAN KUTZ [Opens in a new window]
CRAIG GIN
Affiliation:
Department of Applied Mathematics, University of Washington, Seattle, WA98195, USA emails: crgin@ncsu.edu; kutz@uw.edu
BETHANY LUSCH
Affiliation:
Argonne Leadership Computing Facility, Argonne National Laboratory, Lemont, IL60439, USA email: blusch@anl.gov
STEVEN L. BRUNTON
Affiliation:
Department of Applied Mathematics, University of Washington, Seattle, WA98195, USA emails: crgin@ncsu.edu; kutz@uw.edu Department of Mechanical Engineering, University of Washington, Seattle, WA98195, USA email: sbrunton@uw.edu
J. NATHAN KUTZ
Affiliation:
Department of Applied Mathematics, University of Washington, Seattle, WA98195, USA emails: crgin@ncsu.edu; kutz@uw.edu
Corresponding
E-mail address:
crgin@ncsu.edu
kutz@uw.edu
blusch@anl.gov
sbrunton@uw.edu
Get access
Rights & Permissions[Opens in a new window]

Abstract

We develop a deep autoencoder architecture that can be used to find a coordinate transformation which turns a non-linear partial differential equation (PDE) into a linear PDE. Our architecture is motivated by the linearising transformations provided by the Cole–Hopf transform for Burgers’ equation and the inverse scattering transform for completely integrable PDEs. By leveraging a residual network architecture, a near-identity transformation can be exploited to encode intrinsic coordinates in which the dynamics are linear. The resulting dynamics are given by a Koopman operator matrix K. The decoder allows us to transform back to the original coordinates as well. Multiple time step prediction can be performed by repeated multiplication by the matrix K in the intrinsic coordinates. We demonstrate our method on a number of examples, including the heat equation and Burgers’ equation, as well as the substantially more challenging Kuramoto–Sivashinsky equation, showing that our method provides a robust architecture for discovering linearising transforms for non-linear PDEs.

Keywords

Koopman theory deep neural networks residual networks linearising transforms Cole–Hopf transform

MSC classification

Primary: 35A22: Transform methods (e.g. integral transforms)
Secondary: 35A35: Theoretical approximation to solutions 37M99: None of the above, but in this section 65P99: None of the above, but in this section 68T99: None of the above, but in this section
Type
Papers
Information
European Journal of Applied Mathematics , First View , pp. 1 - 25
Check for updates
Copyright
© The Author(s), 2020. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below.
If you should have access and can't see this content please contact technical support.

References

Ablowitz, M. J. & Segur, H. (1981) Solitons and the Inverse Scattering Transform, Vol. 4, SIAM, Philadelphia, PA.CrossRefGoogle Scholar
Bishop, C. (2006) Pattern Recognition and Machine Learning, Springer, New York, NY.Google Scholar
Boyce, W. E. & DiPrima, R. C. (2008) Elementary Differential Equations, 9th ed., Wiley, Hoboken, NJ.Google Scholar
Brunton, S. L., Brunton, B. W., Proctor, J. L. & Kutz, J. N. (2016) Koopman invariant subspaces and finite linear representations of nonlinear dynamical systems for control. PLOS ONE 11, 1–19.CrossRefGoogle ScholarPubMed
Brunton, S. L. & Kutz, J. N. (2019) Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control, Cambridge University Press, Cambridge.CrossRefGoogle Scholar
Budišić, M. & Mezić, I. (2012) Geometry of the ergodic quotient reveals coherent structures in flows. Physica D Nonlinear Phenomena 241, 1255–1269.CrossRefGoogle Scholar
Champion, K., Lusch, B., Kutz, J. N. & Brunton, S. L. (2019) Data-driven discovery of coordinates and governing equations. Proc. Nat. Acad. Sci. 116, 22445–22451.Google Scholar
Cole, J. D. (1951) On a quasi-linear parabolic equation occurring in aerodynamics. Quart. Appl. Math. 9, 225–236.CrossRefGoogle Scholar
Cybenko, G. (1989) Approximation by superpositions of a sigmoidal function. Math. Control Sig. Syst. (MCSS) 2, 303–314.CrossRefGoogle Scholar
Dsilva, C. J., Talmon, R., Coifman, R. R. & Kevrekidis, I. G. (2018) Parsimonious representation of nonlinear dynamical systems through manifold learning: a chemotaxis case study. Appl. Comput. Harmonic Anal. 44, 759–773.CrossRefGoogle Scholar
Foias, C., Jolly, M., Kevrekidis, I. & Titi, E. (1994) On some dissipative fully discrete nonlinear Galerkin schemes for the Kuramoto-Sivashinsky equation. Phys. Lett. A 186, 87–96.CrossRefGoogle Scholar
Gonzalez-Garcia, R., Rico-Martinez, R. & Kevrekidis, I. (1998) Identification of distributed parameter systems: a neural net based approach. Comput. Chem. Eng. 22, S965–S968.CrossRefGoogle Scholar
Goodfellow, I., Bengio, Y. & Courville, A. (2016) Deep Learning, MIT Press, Cambridge, MA. http://www.deeplearningbook.org.Google Scholar
Haberman, R. (2004) Applied Partial Differential Equations: with Fourier Series and Boundary Value Problems, Pearson Prentice Hall, Upper Saddle River, NJ.Google Scholar
He, K., Zhang, X., Ren, S. & Sun, J. (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Ccomputer Vision and Pattern Recognition, pp. 770–778.CrossRefGoogle Scholar
Hopf, E. (1950) The partial differential equation u t+ uu x= μu xx. Comm. Pure App. Math. 3, 201–230.CrossRefGoogle Scholar
Hornik, K., Stinchcombe, M. & White, H. (1990) Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks. Neural Networks 3, 551–560.CrossRefGoogle Scholar
Kassam, A. & Trefethen, L. (2005) Fourth-order time stepping for stiff PDEs. SIAM J. Sci. Comput. 26, 1214–1233.CrossRefGoogle Scholar
Klus, S., Nüske, F., Koltai, P., Wu, H., Kevrekidis, I., Schütte, C. & Noé, F. (2018) Data-driven model reduction and transfer operator approximation. J. Nonlinear Sci. 28, 985–1010.CrossRefGoogle Scholar
Koopman, B. O. (1931) Hamiltonian systems and transformation in Hilbert space. Proc. Nat. Acad. Sci. 17, 315–318.Google Scholar
Kutz, J. N. (2013) Data-Driven Modeling & Scientific Computation: Methods for Complex Systems & Big Data, Oxford University Press, Oxford.Google Scholar
Kutz, J. N., Brunton, S. L., Brunton, B. W. & Proctor, J. L. (2016) Dynamic Mode Decomposition: Data-Driven Modeling of Complex Systems, SIAM, Philadelphia, PA.CrossRefGoogle Scholar
Kutz, J. N., Proctor, J. L. & Brunton, S. L. (2018) Applied Koopman theory for partial differential equations and data-driven modeling of spatio-temporal systems. Complexity 2018, 1–16.CrossRefGoogle Scholar
Li, Q., Dietrich, F., Bollt, E. M. & Kevrekidis, I. G. (2017) Extended dynamic mode decomposition with dictionary learning: a data-driven adaptive spectral decomposition of the Koopman operator. Chaos Interdiscip. J. Nonlinear Sci. 27, 103111.CrossRefGoogle ScholarPubMed
Lu, L., Shin, S., Su, Y. & Karniadakis, G. (2019) Dying ReLU and initialization: theory and numerical examples. arXiv:1903.06733.Google Scholar
Lu, L., Su, Y. & Karniadakis, G. (2018) Collapse of deep and narrow neural nets. arXiv:1808.04947.Google Scholar
Lusch, B., Kutz, J. N. & Brunton, S. L. (2018) Deep learning for universal linear embeddings of nonlinear dynamics. Nat. Commun. 9, 4950.CrossRefGoogle ScholarPubMed
Mallat, S. (2016) Understanding deep convolutional networks. Philos. Trans. R. Soc. A Math. Phys. Eng. Sci. 374, 20150203.CrossRefGoogle ScholarPubMed
Mardt, A., Pasquali, L., Wu, H. & Noé, F. (2018) VAMPnets: deep learning of molecular kinetics. Nat. Commun. 9, 5.CrossRefGoogle ScholarPubMed
Mezić, I. (2005) Spectral properties of dynamical systems, model reduction and decompositions. Nonlinear Dyn. 41, 309–325.CrossRefGoogle Scholar
Mezić, I. (2013) Analysis of fluid flows via spectral properties of the Koopman operator. Ann. Rev. Fluid Mech. 45, 357–378.CrossRefGoogle Scholar
Mezić, I. & Banaszuk, A. (2004) Comparison of systems with complex behavior. Physica D Nonlinear Phenomena 197, 101–133.CrossRefGoogle Scholar
Neu, J. C. (1980) The method of near-identity transformations and its applications. SIAM J. Appl. Math. 38, 189–208.CrossRefGoogle Scholar
Noé, F. & Nüske, F. (2013) A variational approach to modeling slow processes in stochastic dynamical systems. Multiscale Model. Simul. 11, 635–655.CrossRefGoogle Scholar
Nüske, F., Keller, B. G., Pérez-Hernández, G., Mey, A. S. & Noé, F. (2014) Variational approach to molecular kinetics. J. Chem. Theory Comput. 10, 1739–1752.CrossRefGoogle ScholarPubMed
Otto, S. E. & Rowley, C. W. (2019) Linearly-recurrent autoencoder networks for learning dynamics. SIAM J. Appl. Dyn. Syst. 18, 558–593.CrossRefGoogle Scholar
Page, J. & Kerswell, R. R. (2018) Koopman analysis of burgers equation. Phys. Rev. Fluids 3, 071901.CrossRefGoogle Scholar
Pan, S. & Duraisamy, K. (2020) Physics-informed probabilistic learning of linear embeddings of nonlinear dynamics with guaranteed stability. SIAM J. Appl. Dyn. Syst. 19, 480–509.CrossRefGoogle Scholar
Rico-Martinez, R., Kevrekidis, I. & Krischer, K. (1995) Nonlinear system identification using neural networks: dynamics and instabilities. In: Neural Networks for Chemical Engineers, pp. 409–442.Google Scholar
Rowley, C. W., Mezić, I., Bagheri, S., Schlatter, P. & Henningson, D. (2009) Spectral analysis of nonlinear flows. J. Fluid Mech. 645, 115–127.CrossRefGoogle Scholar
Schmid, P. J. (2010) Dynamic mode decomposition of numerical and experimental data. J. Fluid Mech. 656, 5–28.CrossRefGoogle Scholar
Takeishi, N., Kawahara, Y. & Yairi, T. (2017) Learning Koopman invariant subspaces for dynamic mode decomposition. In: Advances in Neural Information Processing Systems, pp. 1130–1140.Google Scholar
Wehmeyer, C. & Noé, F. (2017) Time-lagged autoencoders: deep learning of slow collective variables for molecular kinetics. J. Chem. Phys. 148, 241703.CrossRefGoogle Scholar
Wiggins, S. (2003) Introduction to Applied Nonlinear Dynamical Systems and Chaos, Vol. 2, Springer, New York, NY.Google Scholar
Williams, M. O., Kevrekidis, I. G. & Rowley, C. W. (2015) A data-driven approximation of the Koopman operator: extending dynamic mode decomposition. J. Nonlinear Sci. 25, 1307–1346.CrossRefGoogle Scholar
Williams, M. O., Rowley, C. W. & Kevrekidis, I. G. (2015) A kernel-based method for data-driven Koopman spectral analysis. J. Comput. Dyn. 2, 247–265.CrossRefGoogle Scholar
Yeung, E., Kundu, S. & Hodas, N. (2019) Learning deep neural network representations for Koopman operators of nonlinear dynamical systems. In: 2019 American Control Conference (ACC), pp. 4832–4839.CrossRefGoogle Scholar

Altmetric attention score

Full text views

Full text views reflects PDF downloads, PDFs sent to Google Drive, Dropbox and Kindle and HTML full text views.

Total number of HTML views: 1339
Total number of PDF views: 101 *
View data table for this chart

* Views captured on Cambridge Core between 24th September 2020 - 25th January 2021. This data will be updated every 24 hours.

Hostname: page-component-898fc554b-mscqj Total loading time: 0.298 Render date: 2021-01-25T21:27:24.220Z Query parameters: { "hasAccess": "0", "openAccess": "0", "isLogged": "0", "lang": "en" } Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "metricsAbstractViews": false, "figures": false, "newCiteModal": false }

Send article to Kindle

To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Deep learning models for global coordinate transformations that linearise PDEs
Available formats
×

Send article to Dropbox

To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

Deep learning models for global coordinate transformations that linearise PDEs
Available formats
×

Send article to Google Drive

To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

Deep learning models for global coordinate transformations that linearise PDEs
Available formats
×
×

Reply to: Submit a response

Your details

Conflicting interests

Do you have any conflicting interests? *