Abstract
Deep learning has significantly improved the state-of-the-art in computer vision and natural language processing, and holds great potential to design effective tools for predicting and simulating complex engineering systems. In particular, scientific machine learning seeks to apply the power of deep learning to scientific and engineering tasks, with operator learning (OL) emerging as a particularly effective tool. OL can approximate nonlinear operators arising in complex engineering systems, making it useful for simulating, designing, and controlling those systems. In this position paper, we provide a comprehensive overview of OL, including its potential applications to complex engineering domains. We cover three variations of OL approaches: deterministic OL for modeling nonautonomous systems, OL with uncertainty quantification (UQ) capabilities, and multifidelity OL. For each variation, we discuss drawbacks and potential applications to engineering, in addition to providing a detailed explanation. We also highlight how multifidelity OL approaches with UQ capabilities can be used to design, optimize, and control engineering systems. Finally, we outline some potential challenges for OL within the engineering domain.
References
1.
Efendiev
, Y.
, Leung
, W. T.
, Lin
, G.
, and Zhang
, Z.
, 2022
, “Efficient Hybrid Explicit-Implicit Learning for Multiscale Problems
,” J. Comput. Phys.
, 467
, p. 111326
.2.
Qin
, T.
, Chen
, Z.
, Jakeman
, J. D.
, and Xiu
, D.
, 2021
, “Data-Driven Learning of Nonautonomous Systems
,” SIAM J. Sci. Comput.
, 43
(3
), pp. A1607
–A1624
. 3.
Brunton
, S. L.
, Proctor
, J. L.
, and Kutz
, J. N.
, 2016
, “Discovering Governing Equations From Data by Sparse Identification of Nonlinear Dynamical Systems
,” Proc. Natl. Acad. Sci. U. S. A.
, 113
(15
), pp. 3932
–3937
. 4.
Brunton
, S. L.
, Proctor
, J. L.
, and Kutz
, J. N.
, 2016
, “Sparse Identification of Nonlinear Dynamics With Control (sindyc)
,” IFAC-PapersOnLine
, 49
(18
), pp. 710
–715
. 5.
Schaeffer
, H.
, 2017
, “Learning Partial Differential Equations Via Data Discovery and Sparse Optimization
,” Proc. R. Soc. A: Math., Phys. Eng. Sci.
, 473
(2197
), p. 20160446
. 6.
Raissi
, M.
, Perdikaris
, P.
, and Karniadakis
, G. E.
, 2019
, “Physics-Informed Neural Networks: A Deep Learning Framework for Solving Forward and Inverse Problems Involving Nonlinear Partial Differential Equations
,” J. Comput. Phys.
, 378
, pp. 686
–707
. 7.
Moya
, C.
, and Lin
, G.
, 2023
, “DAE-PINN: A Physics-Informed Neural Network Model for Simulating Differential Algebraic Equations With Application to Power Networks
,” Neural Computi. Appl.
, 35
(5
), pp. 1
–16
.8.
Karniadakis
, G. E.
, Kevrekidis
, I. G.
, Lu
, L.
, Perdikaris
, P.
, Wang
, S.
, and Yang
, L.
, 2021
, “Physics-Informed Machine Learning
,” Nat. Rev. Phys.
, 3
(6
), pp. 422
–440
. 9.
Qin
, T.
, Wu
, K.
, and Xiu
, D.
, 2019
, “Data Driven Governing Equations Approximation Using Deep Neural Networks
,” J. Comput. Phys.
, 395
, pp. 620
–635
. 10.
Raissi
, M.
, Perdikaris
, P.
, and Karniadakis
, G. E.
, 2018
, “Multistep Neural Networks for Data-Driven Discovery of Nonlinear Dynamical Systems
,” arXiv preprint arXiv:1801.01236.11.
Misyris
, G. S.
, Venzke
, A.
, and Chatzivasileiadis
, S.
, 2020
, “Physics-Informed Neural Networks for Power Systems
,” 2020 IEEE Power & Energy Society General Meeting (PESGM)
, Montreal, QC, Canada
, Aug. 2–6
, IEEE
, pp. 1
–5
. 12.
Lu
, L.
, Jin
, P.
, Pang
, G.
, Zhang
, Z.
, and Karniadakis
, G. E.
, 2021
, “Learning Nonlinear Operators Via Deeponet Based on the Universal Approximation Theorem of Operators
,” Nat. Mach. Intell.
, 3
(3
), pp. 218
–229
. 13.
Chen
, T.
, and Chen
, H.
, 1995
, “Universal Approximation to Nonlinear Operators by Neural Networks With Arbitrary Activation Functions and Its Application to Dynamical Systems
,” IEEE Trans. Neural Netw.
, 6
(4
), pp. 911
–917
.14.
Li
, G.
, Moya
, C.
, and Zhang
, Z.
, 2022
, “On Learning the Dynamical Response of Nonlinear Control Systems With Deep Operator Networks
,” arXiv preprint arXiv:2206.06536.15.
Moya
, C.
, Zhang
, S.
, Yue
, M.
, and Lin
, G.
, 2023
, “Deeponet-Grid-UQ: A Trustworthy Deep Operator Framework for Predicting the Power Grid’s Post-Fault Trajectories
,” Neurocomputing
, 535
, pp. 166
–182
.16.
Cai
, S.
, Wang
, Z.
, Lu
, L.
, Zaki
, T. A.
, and Karniadakis
, G. E.
, 2021
, “Deepm&Mnet: Inferring the Electroconvection Multiphysics Fields Based on Operator Approximation by Neural Networks
,” J. Comput. Phys.
, 436
, p. 110296
. 17.
Wang
, S.
, and Perdikaris
, P.
, 2021
, “Long-Time Integration of Parametric Evolution Equations With Physics-Informed Deeponets
,” J. Comput. Phys.
, 475
, p. 111855
.18.
Lin
, G.
, Moya
, C.
, and Zhang
, Z.
, 2021
, “Accelerated Replica Exchange Stochastic Gradient Langevin Diffusion Enhanced Bayesian Deeponet for Solving Noisy Parametric PEDS
,” arXiv preprint arXiv:2111.02484.19.
Yang
, Y.
, Kissas
, G.
, and Perdikaris
, P.
, 2022
, “Scalable Uncertainty Quantification for Deep Operator Networks Using Randomized Priors
,” arXiv preprint arXiv:2203.03048.20.
Li
, Z.
, Kovachki
, N.
, Azizzadenesheli
, K.
, Liu
, B.
, Bhattacharya
, K.
, Stuart
, A.
, and Anandkumar
, A.
, 2020
, “Fourier Neural Operator for Parametric Partial Differential Equations
,” arXiv preprint arXiv:2010.08895.21.
Kovachki
, N.
, Li
, Z.
, Liu
, B.
, Azizzadenesheli
, K.
, Bhattacharya
, K.
, Stuart
, A.
, and Anandkumar
, A.
, 2021
, “Neural Operator: Learning Maps Between Function Spaces
,” arXiv preprint arXiv:2108.08481.22.
Zhang
, Z.
, Leung
, W. T.
, and Schaeffer
, H.
, 2022
, “Belnet: Basis Enhanced Learning, a Mesh-Free Neural Operator
,” arXiv preprint arXiv:2212.07336.23.
Moya
, C.
, Lin
, G.
, Zhao
, T.
, and Yue
, M.
, 2023
, “On Approximating the Dynamic Response of Synchronous Generators via Operator Learning: A Step Towards Building Deep Operator-Based Power Grid Simulators
,” arXiv preprint arXiv:2301.12538.24.
Cui
, W.
, Yang
, W.
, and Zhang
, B.
, 2023
, “A Frequency Domain Approach to Predict Power System Transients
,” IEEE Trans. Power Syst.
, pp. 1
–13
.25.
Kasahara
, H.
, Fujii
, H.
, and Iwata
, M.
, 1987
, “Parallel Processing of Robot Motion Simulation
,” IFAC Proceedings Vol.
, 20
(5
), pp. 329
–336
. 26.
Choi
, H.
, Crump
, C.
, Duriez
, C.
, Elmquist
, A.
, Hager
, G.
, Han
, D.
, Hearl
, F.
, Hodgins
, J.
, Jain
, A.
, Leve
, F.
, and Li
, C.
, 2021
, “On the Use of Simulation in Robotics: Opportunities, Challenges, and Suggestions for Moving Forward
,” Proc. Natl. Acad. Sci. U. S. A.
, 118
(1
), p. e1907856118
. 27.
Henrich
, D.
, 1997
, “Fast Motion Planning by Parallel Processing—A Review
,” J. Intell. Rob. Syst.
, 20
(1
), pp. 45
–69
. 28.
Negrut
, D.
, Serban
, R.
, Mazhar
, H.
, and Heyn
, T.
, 2014
, “Parallel Computing in Multibody System Dynamics: Why, When, and How
,” J. Comput. Nonlinear. Dyn.
, 9
(4
), p. 041007
. 29.
Coelho
, P.
, and Nunes
, U.
, 2005
, “Path-Following Control of Mobile Robots in Presence of Uncertainties
,” IEEE Trans. Rob.
, 21
(2
), pp. 252
–261
. 30.
Iserles
, A.
, 2009
, A First Course in the Numerical Analysis of Differential Equations
, Vol. 44
, Cambridge University Press
, Cambridge, UK
.31.
Jin
, P.
, Meng
, S.
, and Lu
, L.
, 2022
, “Mionet: Learning Multiple-Input Operators Via Tensor Product
,” SIAM J. Sci. Comput.
, 44
(6
), pp. A3490
–A3514
. 32.
Sun
, Y.
, Moya
, C.
, Lin
, G.
, and Yue
, M.
, 2022
, “Deepgraphonet: A Deep Graph Operator Network to Learn and Zero-Shot Transfer the Dynamic Response of Networked Systems
,” arXiv preprint arXiv:2209.10622.33.
Michałowska
, K.
, Goswami
, S.
, Karniadakis
, G. E.
, and Riemer-Sørensen
, S.
, 2023
, “Neural Operator Learning for Long-Time Integration in Dynamical Systems With Recurrent Neural Networks
,” arXiv preprint arXiv:2303.02243.34.
Moya
, C.
, and Lin
, G.
, 2022
, “Fed-deeponet: Stochastic Gradient-Based Federated Training of Deep Operator Networks
,” Algorithms
, 15
(9
), p. 325
. Copyright © 2023 by ASME
You do not currently have access to this content.
