Skip to main content Accessibility help
×
Home
Hostname: page-component-77ffc5d9c7-866w8 Total loading time: 0.551 Render date: 2021-04-23T12:42:41.496Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "metricsAbstractViews": false, "figures": false, "newCiteModal": false, "newCitedByModal": true }

Learning and meta-learning of stochastic advection–diffusion–reaction systems from sparse measurements

Published online by Cambridge University Press:  15 June 2020

XIAOLI CHEN
Affiliation:
Center for Mathematical Sciences and School of Mathematics and Statistics, Huazhong University of Science and Technology, Wuhan430074, China email: xlchen@hust.edu.cn Division of Applied Mathematics, Brown University, Providence, RI02912, USA email: george_karniadakis@brown.edu
JINQIAO DUAN
Affiliation:
Department of Applied Mathematics, Illinois Institute of Technology, Chicago, IL60616, USA email: duan@iit.edu
GEORGE EM KARNIADAKIS
Affiliation:
Division of Applied Mathematics, Brown University, Providence, RI02912, USA email: george_karniadakis@brown.edu Pacific Northwest National Laboratory, Richland, WA99354, USA

Abstract

Physics-informed neural networks (PINNs) were recently proposed in [18] as an alternative way to solve partial differential equations (PDEs). A neural network (NN) represents the solution, while a PDE-induced NN is coupled to the solution NN, and all differential operators are treated using automatic differentiation. Here, we first employ the standard PINN and a stochastic version, sPINN, to solve forward and inverse problems governed by a non-linear advection–diffusion–reaction (ADR) equation, assuming we have some sparse measurements of the concentration field at random or pre-selected locations. Subsequently, we attempt to optimise the hyper-parameters of sPINN by using the Bayesian optimisation method (meta-learning) and compare the results with the empirically selected hyper-parameters of sPINN. In particular, for the first part in solving the inverse deterministic ADR, we assume that we only have a few high-fidelity measurements, whereas the rest of the data is of lower fidelity. Hence, the PINN is trained using a composite multi-fidelity network, first introduced in [12], that learns the correlations between the multi-fidelity data and predicts the unknown values of diffusivity, transport velocity and two reaction constants as well as the concentration field. For the stochastic ADR, we employ a Karhunen–Loève (KL) expansion to represent the stochastic diffusivity, and arbitrary polynomial chaos (aPC) to represent the stochastic solution. Correspondingly, we design multiple NNs to represent the mean of the solution and learn each aPC mode separately, whereas we employ a separate NN to represent the mean of diffusivity and another NN to learn all modes of the KL expansion. For the inverse problem, in addition to stochastic diffusivity and concentration fields, we also aim to obtain the (unknown) deterministic values of transport velocity and reaction constants. The available data correspond to 7 spatial points for the diffusivity and 20 space–time points for the solution, both sampled 2000 times. We obtain good accuracy for the deterministic parameters of the order of 1–2% and excellent accuracy for the mean and variance of the stochastic fields, better than three digits of accuracy. In the second part, we consider the previous stochastic inverse problem, and we use Bayesian optimisation to find five hyper-parameters of sPINN, namely the width, depth and learning rate of two NNs for learning the modes. We obtain much deeper and wider optimal NNs compared to the manual tuning, leading to even better accuracy, i.e., errors less than 1% for the deterministic values, and about an order of magnitude less for the stochastic fields.

MSC classification

Type
Papers
Copyright
© The Author(s), 2020. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below.

References

Barajas-Solano, D. A. & Tartakovsky, A. M. (2019) Approximate bayesian model inversion for pdes with heterogeneous and state-dependent coefficients. J. Comput. Phys. 395, 247262.CrossRefGoogle Scholar
Bergstra, J. S., Bardenet, R., Bengio, Y. & Kégl, B. (2011) Algorithms for hyper-parameter optimization. In: Advances in Neural Information Processing Systems, pp. 25462554.Google Scholar
Chen, T. Q., Rubanova, Y., Bettencourt, J. & Duvenaud, D. K. (2018) Neural ordinary differential equations. In: Advances in Neural Information Processing Systems, pp. 65716583.Google Scholar
Chaudhari, P., Oberman, A., Osher, S., Soatto, S. & Carlier, G. (2018) Deep relaxation: partial differential equations for optimizing deep neural networks. Res. Math. Sci. 5 (3), 30.CrossRefGoogle Scholar
Falkner, S., Klein, A. & Hutter, F. (2018) BOHB: robust and efficient hyperparameter optimization at scale. arXiv preprint arXiv:1807.01774.Google Scholar
Finn, C., Abbeel, P. & Levine, S. (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In: Proceedings of the 34th International Conference on Machine Learning-Volume 70, JMLR.org, pp. 11261135.Google Scholar
Han, J., Jentzen, A. & Weinan, E. (2018) Solving high-dimensional partial differential equations using deep learning. Proc. Natl. Acad. Sci. 115, 85058510.CrossRefGoogle ScholarPubMed
He, Y., Lin, J., Liu, Z., Wang, H., Li, L.-J. & Han, S. (2018) AMC: AutoML for model compression and acceleration on mobile devices. In: Proceedings of the European Conference on Computer Vision (ECCV),pp. 784800.CrossRefGoogle Scholar
Jaafra, Y., Laurent, J. L., Deruyver, A. & Naceur, M. S. (2018) A review of meta-reinforcement learning for deep neural networks architecture search. arXiv preprint arXiv:1812.07995.Google Scholar
Li, L., Jamieson, K., DeSalvo, G., Rostamizadeh, A. & Talwalkar, A. (2018) Hyperband: a novel bandit-based approach to hyperparameter optimization. J. Mach. Learn. Res. 18, 151.Google Scholar
Li, Y. & Osher, S. (2009) Coordinate descent optimization for l1 minimization with application to compressed sensing; a greedy algorithm. Inverse Problemsd Imaging 3, 487503.CrossRefGoogle Scholar
Meng, X. & Karniadakis, G. E. (2020) A composite neural network that learns from multi-fidelity data: application to function approximation and inverse PDE problems. J. Comput. Phys. 401, 109020.CrossRefGoogle Scholar
Mitchell, M. (1998) An Introduction to Genetic Algorithms, MIT Press.CrossRefGoogle Scholar
Pang, G., Lu, L. & Karniadakis, G. E. (2019) fpinns: fractional physics-informed neural networks. SIAM J. Sci. Comput. 41, A2603A2626.CrossRefGoogle Scholar
Pang, G., Yang, L. & Karniadakis, G. E. (2019) Neural-net-induced gaussian process regression for function approximation and pde solution. J. Comput. Phys. 384, 270288.CrossRefGoogle Scholar
Paulson, J. A., Buehler, E. A. & Mesbah, A. (2017) Arbitrary polynomial chaos for uncertainty propagation of correlated random variables in dynamic systems. IFAC-PapersOnLine 50, 35483553.CrossRefGoogle Scholar
Qin, T., Wu, K. & Xiu, D. (2019) Data driven governing equations approximation using deep neural networks. J. Comput. Phys. 395, 620635.CrossRefGoogle Scholar
Raissi, M., Perdikaris, P. & Karniadakis, G. E. (2019) Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J. Comput. Phys. 378, 686707.CrossRefGoogle Scholar
Raissi, M., Perdikaris, P. & Karniadakis, G. E. (2018) Numerical gaussian processes for time-dependent and nonlinear partial differential equations. SIAM J. Sci. Comput. 40, A172A198.CrossRefGoogle Scholar
Sirignano, J. & Spiliopoulos, K. (2018) Dgm: a deep learning algorithm for solving partial differential equations. J. Comput. Phys. 375, 13391364.CrossRefGoogle Scholar
Snoek, J., Larochelle, H. & Adams, R. P. (2012) Practical bayesian optimization of machine learning algorithms. Adv. Neural Inf. Process Syst. 25, 29512959.Google Scholar
Snoek, J., Rippel, O., Swersky, K., Kiros, R., Satish, N., Sundaram, N., Patwary, M., Prabhat, M. & Adams, R. (2015) Scalable bayesian optimization using deep neural networks. Int. Conf. Mach. Learn. 37, 21712180.Google Scholar
Tartakovsky, A. M., Marrero, C. O., Tartakovsky, D. & Barajas-Solano, D. (2018) Learning parameters and constitutive relationships with physics informed deep neural networks. arXiv preprint arXiv:1808.03398.Google Scholar
Wan, X. & Karniadakis, G. E. (2006) Multi-element generalized polynomial chaos for arbitrary probability measures. SIAM J. Sci. Comput. 28, 901928.CrossRefGoogle Scholar
Zhang, D., Lu, L., Guo, L. & Karniadakis, G. E. (2019) Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems, J. Comput. Phys. 397, 108850.CrossRefGoogle Scholar

Full text views

Full text views reflects PDF downloads, PDFs sent to Google Drive, Dropbox and Kindle and HTML full text views.

Total number of HTML views: 0
Total number of PDF views: 59 *
View data table for this chart

* Views captured on Cambridge Core between 15th June 2020 - 23rd April 2021. This data will be updated every 24 hours.

Send article to Kindle

To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Learning and meta-learning of stochastic advection–diffusion–reaction systems from sparse measurements
Available formats
×

Send article to Dropbox

To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

Learning and meta-learning of stochastic advection–diffusion–reaction systems from sparse measurements
Available formats
×

Send article to Google Drive

To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

Learning and meta-learning of stochastic advection–diffusion–reaction systems from sparse measurements
Available formats
×
×

Reply to: Submit a response


Your details


Conflicting interests

Do you have any conflicting interests? *