Abstract

While most calibration methods focus on inferring a set of model parameters that are unknown but assumed to be constant, many models have parameters that have a functional relation with the controllable input variables. Formulating a low-dimensional approximation of these calibration functions allows modelers to use low-fidelity models to explore phenomena at lengths and time scales unattainable with their high-fidelity sources. While functional calibration methods are available for low-dimensional problems (e.g., one to three unknown calibration functions), exploring high-dimensional spaces of unknown calibration functions (e.g., more than ten) is still a challenging task due to its computational cost and the risk for identifiability issues. To address this challenge, we introduce a semiparametric calibration method that uses an approximate Bayesian computation scheme to quantify the uncertainty in the unknown calibration functions and uses this insight to identify what functions can be replaced with low-dimensional approximations. Through a test problem and a coarse-grained model of an epoxy resin, we demonstrate that the introduced method enables the identification of a low-dimensional set of calibration functions with a limited compromise in calibration accuracy. The novelty of the presented method is the ability to synthesize domain knowledge from various sources (i.e., physical experiments, simulation models, and expert insight) to enable high-dimensional functional calibration without the need for prior knowledge on the class of unknown calibration functions.

References

1.
Higdon
,
D.
,
Gattiker
,
J.
,
Williams
,
B.
, and
Rightley
,
M.
,
2008
, “
Computer Model Calibration Using High-Dimensional Output
,”
J. Am. Stat. Assoc.
,
103
(
482
), pp.
570
583
.10.1198/016214507000000888
2.
Leoni
,
N.
, and
Amon
,
C. H.
,
2000
, “
Bayesian Surrogates for Integrating Numerical, Analytical, and Experimental Data: Application to Inverse Heat Transfer in Wearable Computers
,”
IEEE Trans. Compon. Packaging Technol.
,
23
(
1
), pp.
23
32
.10.1109/6144.833038
3.
Higdon
,
D.
,
Kennedy
,
M.
,
Cavendish
,
J. C.
,
Cafeo
,
J. A.
, and
Ryne
,
R. D.
,
2004
, “
Combining Field Data and Computer Simulations for Calibration and Prediction
,”
SIAM J. Sci. Comput.
,
26
(
2
), pp.
448
466
.10.1137/S1064827503426693
4.
Gattiker
,
J.
,
Higdon
,
D.
,
Keller-McNulty
,
S.
,
McKay
,
M.
,
Moore
,
L.
, and
Williams
,
B.
,
2006
, “
Combining Experimental Data and Computer Simulations, With an Application to Flyerplate Experiments
,”
Bayesian Anal.
,
1
(
4
), pp.
765
792
.10.1214/06-BA125
5.
Xiong
,
Y.
,
Chen
,
W.
,
Tsui
,
K.-L.
, and
Apley
,
D. W.
,
2009
, “
A Better Understanding of Model Updating Strategies in Validating Engineering Models
,”
Comput. Methods Appl. Mech. Eng.
,
198
(
15–16
), pp.
1327
1337
.10.1016/j.cma.2008.11.023
6.
Li
,
Y. G.
,
2010
, “
Gas Turbine Performance and Health Status Estimation Using Adaptive Gas Path Analysis
,”
ASME J. Eng. Gas Turbines Power
,
132
(
4
), p.
041701
.10.1115/1.3159378
7.
Liu
,
X.
,
Mao
,
K.
,
Wang
,
X.
,
Wang
,
X.
, and
Wang
,
Y.
,
2020
, “
A Modified Quality Loss Model of Service Life Prediction for Products Via Wear Regularity
,”
Reliab. Eng. Syst. Safety
,
204
, p.
107187
.10.1016/j.ress.2020.107187
8.
Li
,
X.
,
Zhang
,
M.
,
Wang
,
Y.
,
Zhang
,
M.
,
Prasad
,
A.
,
Chen
,
W.
,
Schadler
,
L.
, and
Catherine Brinson
,
L.
,
2019
, “
Rethinking Interphase Representations for Modeling Viscoelastic Properties for Polymer Nanocomposites
,”
Materials
,
6
, p.
100277
.10.1016/j.mtla.2019.100277
9.
Xia
,
W.
,
Song
,
J.
,
Hansoge
,
N. K.
,
Phelan
,
F. R.
,
Keten
,
S.
, and
Douglas
,
J. F.
,
2018
, “
Energy Renormalization for Coarse-Graining the Dynamics of a Model Glass-Forming Liquid
,”
J. Phys. Chem. B
,
122
(
6
), pp.
2040
2045
.10.1021/acs.jpcb.8b00321
10.
Sacks
,
J.
,
Welch
,
W. J.
,
Mitchell
,
T. J.
, and
Wynn
,
H. P.
,
1989
, “
Design and Analysis of Computer Experiments
,”
Stat. Sci.
,
4
(
4
), pp.
409
423
.10.1214/ss/1177012413
11.
Han
,
G.
,
Santner
,
T. J.
, and
Rawlinson
,
J. J.
,
2009
, “
Simultaneous Determination of Tuning and Calibration Parameters for Computer Experiments
,”
Technometrics
,
51
(
4
), pp.
464
474
.10.1198/TECH.2009.08126
12.
Kennedy
,
M. C.
, and
O'Hagan
,
A.
,
2001
, “
Bayesian Calibration of Computer Models
,”
J. R. Stat. Soc.: Ser. B (Stat. Methodology)
,
63
(
3
), pp.
425
464
.10.1111/1467-9868.00294
13.
Qian
,
P. Z. G.
, and
Jeff Wu
,
C. F.
,
2008
, “
Bayesian Hierarchical Modeling for Integrating Low-Accuracy and High-Accuracy Experiments
,”
Technometrics
,
50
(
2
), pp.
192
204
.10.1198/004017008000000082
14.
Plumlee
,
M.
,
2017
, “
Bayesian Calibration of Inexact Computer Models
,”
J. Am. Stat. Assoc.
,
112
(
519
), pp.
1274
1285
.10.1080/01621459.2016.1211016
15.
Fugate
,
M.
,
Williams
,
B.
,
Higdon
,
D.
,
Hanson
,
K. M.
,
Gattiker
,
J.
,
Chen
,
S. R.
,
Unal
,
C.
,
Al
,
P.
,
Du
,
B.
, and
U-Nb
,
T.
,
2010
, “
Hierarchical Bayesian Analysis and the Preston-Tonks-Wallace Model
,” accessed Aug. 9, 2021, https://kmh-lanl.hansonhub.com/publications/larept05.pdf
16.
Ezzat
,
A. A.
,
Pourhabib
,
A.
, and
Ding
,
Y.
,
2018
, “
Sequential Design for Functional Calibration of Computer Models
,”
Technometrics
,
60
(
3
), pp.
286
296
.10.1080/00401706.2017.1377638
17.
Tao
,
S.
,
Apley
,
D. W.
,
Chen
,
W.
,
Garbo
,
A.
,
Pate
,
D. J.
, and
German
,
B. J.
,
2019
, “
Input Mapping for Model Calibration With Application to Wing Aerodynamics
,”
AIAA J.
,
57
(
7
), pp.
2734
2745
.10.2514/1.J057711
18.
Andrew Brown
,
D.
, and
Atamturktur
,
S.
,
2018
, “
Nonparametric Functional Calibration of Computer Models
,”
Statistica Sin.
,
28
(
2
), pp.
721
742
.10.5705/ss.202015.0344
19.
Plumlee
,
M.
,
Roshan Joseph
,
V.
, and
Yang
,
H.
,
2016
, “
Calibrating Functional Parameters in the Ion Channel Models of Cardiac Cells
,”
J. Am. Stat. Assoc.
,
111
(
514
), pp.
500
509
.10.1080/01621459.2015.1119695
20.
Pourhabib
,
A.
, and
Balasundaram
,
B.
,
2015
, “
Non-Isometric Curve to Surface Matching With Incomplete Data for Functional Calibration
,” Machine Learning, e-print arXiv:1508.01240.
21.
Farmanesh
,
B.
,
Pourhabib
,
A.
,
Balasundaram
,
B.
, and
Buchanan
,
A.
,
2021
, “
A Bayesian Framework for Functional Calibration of Expensive Computational Models Through Non-Isometric Matching
,”
IISE Trans.
,
53
(
3
), pp.
352
364
.10.1080/24725854.2020.1774688
22.
Jiang
,
Z.
,
Chen
,
S.
,
Apley
,
D. W.
, and
Chen
,
W.
,
2016
, “
Reduction of Epistemic Model Uncertainty in Simulation-Based Multidisciplinary Design
,”
ASME J. Mech. Des.
,
138
(
8
), p.
081403
.10.1115/1.4033918
23.
Lee
,
G.
,
Kim
,
W.
,
Oh
,
H.
,
Youn
,
B. D.
, and
Kim
,
N. H.
, oct
2019
, “
Review of Statistical Model Calibration and Validation–From the Perspective of Uncertainty Structures
,”
Struct. Multidiscip. Optim.
,
60
(
4
), pp.
1619
1644
.10.1007/s00158-019-02270-2
24.
Duan
,
K.
,
He
,
Y.
,
Li
,
Y.
,
Liu
,
J.
,
Zhang
,
J.
,
Hu
,
Y.
,
Lin
,
R.
,
Wang
,
X.
,
Deng
,
W.
, and
Li
,
L.
,
2019
, “
Machine-Learning Assisted Coarse-Grained Model for Epoxies Over Wide Ranges of Temperatures and Cross-Linking Degrees
,”
Mater. Des.
,
183
, p.
108130
.10.1016/j.matdes.2019.108130
25.
Taka
,
E.
,
Stein
,
S.
, and
Williamson
,
J. H.
,
2020
, “
Increasing Interpretability of Bayesian Probabilistic Programming Models Through Interactive Representations
,”
Front. Comput. Sci.
,
2
, p.
52
.10.3389/fcomp.2020.567344
26.
Picheny
,
V.
,
Ginsbourger
,
D.
,
Richet
,
Y.
, and
Caplin
,
G.
,
2013
, “
Quantile-Based Optimization of Noisy Computer Experiments With Tunable Precision
,”
Technometrics
,
55
(
1
), pp.
2
13
.10.1080/00401706.2012.707580
27.
Le Gratiet
,
L.
, and
Garnier
,
J.
,
2014
, “
Recursive co-Kriging Model for Design of Computer Experiments With Multiple Levels of Fidelity
,”
Int. J. Uncertainty Quantif.
,
4
(
5
), pp.
365
386
.10.1615/Int.J.UncertaintyQuantification.2014006914
28.
Kennedy
,
M.
, and
O'Hagan
,
A.
,
2000
, “
Predicting the Output From a Complex Computer Code When Fast Approximations Are Available
,”
Biometrika
,
87
(
1
), pp.
1
13
.10.1093/biomet/87.1.1
29.
Goh
,
J.
,
Bingham
,
D.
,
Holloway
,
J. P.
,
Grosskopf
,
M. J.
,
Kuranz
,
C. C.
, and
Rutter
,
E.
,
2013
, “
Prediction and Computer Model Calibration Using Outputs From Multifidelity Simulators
,”
Technometrics
,
55
(
4
), pp.
501
512
.10.1080/00401706.2013.838910
30.
Tuo
,
R.
, and
Jeff Wu
,
C. F.
,
2015
, “
Efficient Calibration for Imperfect Computer Models
,”
Ann. Stat.
,
43
(
6
), pp.
2331
2352
.10.1214/15-AOS1314
31.
Atamturktur
,
S.
,
Hegenderfer
,
J.
,
Williams
,
B.
,
Egeberg
,
M.
,
Lebensohn
,
R. A.
, and
Unal
,
C.
,
2015
, “
A Resource Allocation Framework for Experiment-Based Validation of Numerical Models
,”
Mech. Adv. Mater. Struct.
,
22
(
8
), pp.
641
654
.10.1080/15376494.2013.828819
32.
Schaback
,
R.
,
1999
, “
Native Hilbert Spaces for Radial Basis Functions i
,”
New Developments in Approximation Theory
,
M. W.
Müller
,
M. D.
Buhmann
,
D. H.
Mache
, and
M.
Felten
, eds.,
Birkhäuser Basel
,
Basel
, pp.
255
282
.
33.
Schölkopf
,
B.
,
Herbrich
,
R.
, and
Smola
,
A. J.
,
2001
, “
A Generalized Representer Theorem
,”
Computational Learning Theory
,
H.
David
and
W.
Bob
, eds.,
Springer Berlin Heidelberg
,
Berlin, Heidelberg
, pp.
416
426
.
34.
Rasmussen
,
C. E.
, and
Williams
,
C. K. I.
,
2005
,
Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning)
,
The MIT Press
, Cambridge, MA.
35.
Grazian
,
C.
, and
Fan
,
Y.
,
2020
, “
A Review of Approximate Bayesian Computation Methods Via Density Estimation: Inference for Simulator-Models
,”
WIREs Comput. Stat.
,
12
(
4
), p.
e1486
.10.1002/wics.1486
36.
Pavel
,
I.
,
Dmitrii
,
P.
,
Timur
,
G.
,
Dmitry
,
V.
, and
Wilson
,
A. G.
,
2018
, “
Averaging Weights Leads to Wider Optima and Better Generalization
,”
34th Conference on Uncertainty in Artificial Intelligence 2018,
Monterey, CA, Aug. 6, pp.
876
885
.
37.
Maddox
,
W. J.
,
Izmailov
,
P.
,
Garipov
,
T.
,
Vetrov
,
D. P.
, and
Wilson
,
A. G.
,
2019
, “
A Simple Baseline for Bayesian Uncertainty in Deep Learning
,” Proceedings of the 33rd International Conference on Neural Information Processing Systems,
Vancouver, BC, Canada, Dec. 8, pp.
13153
13164
.
38.
Plumlee
,
M.
, March
2014
, “
Fast Prediction of Deterministic Functions Using Sparse Grid Experimental Designs
,”
J. Am. Stat. Assoc.
,
109
(
508
), pp.
1581
1591
.10.1080/01621459.2014.900250
39.
Forrester
,
A. I.
, and
Keane
,
A. J.
,
2009
, “
Recent Advances in Surrogate-Based Optimization
,”
Prog. Aerosp. Sci.
,
45
(
1–3
), pp.
50
79
.10.1016/j.paerosci.2008.11.001
40.
Pritchard
,
J. K.
,
Seielstad
,
M. T.
,
Perez-Lezaun
,
A.
, and
Feldman
,
M. W.
,
1999
, “
Population Growth of Human Y Chromosomes: A Study of Y Chromosome Microsatellites
,”
Mol. Biol. Evol.
,
16
(
12
), pp.
1791
1798
.10.1093/oxfordjournals.molbev.a026091
41.
Marjoram
,
P.
,
Molitor
,
J.
,
Plagnol
,
V.
, and
Tavaré
,
S.
,
2003
, “
Markov Chain Monte Carlo Without Likelihoods
,”
Proc. Natl. Acad. Sci.
,
100
(
26
), pp.
15324
15328
.10.1073/pnas.0306899100
42.
Sisson
,
S. A.
,
Fan
,
Y.
, and
Beaumont
,
M. A.
,
2018
, “
Overview of Approximate Bayesian Computation
,”
arXiv:1802.09720
.10.48550/arXiv.1802.09720
43.
Arora
,
J.
,
2016
,
Introduction to Optimum Design
, 4nd ed.,
Elsevier
, London, UK.
44.
Bertsimas
,
D.
, and
Thiele
,
A.
,
2014
, “
Robust and Data-Driven Optimization: Modern Decision Making Under Uncertainty
,”
INFORMS Tutorials in Operations Research
, pp.
95
122
.
45.
Bertsimas
,
D.
,
Nohadani
,
O.
, and
Teo
,
K. M.
,
2010
, “
Robust Optimization for Unconstrained Simulation-Based Problems
,”
Oper. Res.
,
58
(
1
), pp.
161
178
.10.1287/opre.1090.0715
46.
Saltelli
,
A.
,
Annoni
,
P.
,
Azzini
,
I.
,
Campolongo
,
F.
,
Ratto
,
M.
, and
Tarantola
,
S.
,
2010
, “
Variance Based Sensitivity Analysis of Model Output. design and Estimator for the Total Sensitivity Index
,”
Comput. Physics Commun.
,
181
(
2
), pp.
259
270
.10.1016/j.cpc.2009.09.018
47.
Despotovic
,
M.
,
Nedic
,
V.
,
Despotovic
,
D.
, and
Despotovic
,
S.
,
2016
, “
Evaluation of Empirical Models for Predicting Monthly Mean Horizontal Diffuse Solar Radiation
,”
Renewable Sustainable Energy Rev.
,
56
, pp.
246
260
.10.1016/j.rser.2015.11.058
48.
Göçken
,
M.
,
Özçalici
,
M.
,
Boru
,
A.
, and
Dosdoğru
,
A. T.
,
2016
, “
Integrating Metaheuristics and Artificial Neural Networks for Improved Stock Price Prediction
,”
Expert Syst. Appl.
,
44
, pp.
320
331
.10.1016/j.eswa.2015.09.029
49.
Li
,
M.-F.
,
Tang
,
X.-P.
,
Wu
,
W.
, and
Liu
,
H.-B.
,
2013
, “
General Models for Estimating Daily Global Solar Radiation for Different Solar Radiation Zones in Mainland China
,”
Energy Convers. Manage.
,
70
, pp.
139
148
.10.1016/j.enconman.2013.03.004
50.
Schwarz
,
G.
,
1978
, “
Estimating the Dimension of a Model
,”
Ann. Stat.
,
6
(
2
), pp.
461
464
.10.1214/aos/1176344136
51.
Komarov
,
P. V.
,
Yu-Tsung
,
C.
,
Shih-Ming
,
C.
,
Khalatur
,
P. G.
, and
Reineker
,
P.
,
2007
, “
Highly Cross-Linked Epoxy Resins: An Atomistic Molecular Dynamics Simulation Combined With a Mapping/Reverse Mapping Procedure
,”
Macromolecules
,
40
(
22
), pp.
8104
8113
.10.1021/ma070702+
52.
Bandyopadhyay
,
A.
,
Valavala
,
P. K.
,
Clancy
,
T. C.
,
Wise
,
K. E.
, and
Odegard
,
G. M.
,
2011
, “
Molecular Modeling of Crosslinked Epoxy Polymers: The Effect of Crosslink Density on Thermomechanical Properties
,”
Polymer
,
52
(
11
), pp.
2445
2452
.10.1016/j.polymer.2011.03.052
53.
King
,
J. A.
,
Klimek
,
D. R.
,
Miskioglu
,
I.
, and
Odegard
,
G. M.
,
2013
, “
Mechanical Properties of Graphene Nanoplatelet/Epoxy Composites
,”
J. Appl. Polym. Sci.
,
128
(
6
), pp.
4217
4223
.10.1002/app.38645
54.
Wu
,
C.
, and
Xu
,
W.
,
2007
, “
Atomistic Simulation Study of Absorbed Water Influence on Structure and Properties of Crosslinked Epoxy Resin
,”
Polymer
,
48
(
18
), pp.
5440
5448
.10.1016/j.polymer.2007.06.038
55.
Meng
,
Z.
,
Bessa
,
M. A.
,
Xia
,
W.
,
Liu
,
W. K.
, and
Keten
,
S.
,
2016
, “
Predicting the Macroscopic Fracture Energy of Epoxy Resins From Atomistic Molecular Simulations
,”
Macromolecules
,
49
(
24
), pp.
9474
9483
.10.1021/acs.macromol.6b01508
56.
Li
,
C.
,
Browning
,
A. R.
,
Christensen
,
S.
, and
Strachan
,
A.
,
2012
, “
Atomistic Simulations on Multilayer Graphene Reinforced Epoxy Composites
,”
Compos. Part A: Appl. Sci. Manuf.
,
43
(
8
), pp.
1293
1300
.10.1016/j.compositesa.2012.02.015
57.
Giuntoli
,
A.
,
Hansoge
,
N. K.
,
van Beek
,
A.
,
Meng
,
Z.
,
Chen
,
W.
, and
Keten
,
S.
,
2021
, “
Systematic Coarse-Graining of Epoxy Resins With Machine Learning-Informed Energy Renormalization
,”
NPJ Comput. Mater.
,
7
(
1
), pp.
1
12
.10.1038/s41524-021-00634-1
58.
Sobol
,
I. M.
, May
1976
, “
Uniformly Distributed Sequences With an Additional Uniform Property
,”
USSR Comput. Math. Math. Phys.
,
16
(
5
), pp.
236
242
.10.1016/0041-5553(76)90154-3
59.
van Beek
,
A.
,
Tao
,
S.
, and
Chen
,
W.
,
2019
, “
Global Emulation Through Normative Decision Making and Thrifty Adaptive Batch Sampling
,”
ASME
Paper No. DETC2019-98223.10.1115/DETC2019-98223
60.
van Beek
,
A.
,
Tao
,
S.
,
Plumlee
,
M.
,
Apley
,
D. W.
, and
Chen
,
W.
,
2020
, “
Integration of Normative Decision-Making and Batch Sampling for Global Metamodeling
,”
ASME J. Mech. Des.
,
142
(
3
), p.
031114
.10.1115/1.4045601
61.
Pederson
,
K.
,
Emblemsvåg
,
J.
,
Bailey
,
R.
,
Allen
,
J. K.
, and
Mistree
,
F.
,
2000
, “
Validating Design Methods & Research: The Validation Square
,”
Design Engineering Technical Conferences
, Baltimore, MD, Sept. 10–14, pp.
1
13
.https://cecas.clemson.edu/cedar/wpcontent/uploads/2016/07/2-Mistree2000.pdf
62.
Arendt
,
P. D.
,
Apley
,
D. W.
, and
Chen
,
W.
,
2012
, “
Quantification of Model Uncertainty: Calibration, Model Discrepancy, and Identifiability
,”
ASME J. Mech. Des.
,
134
(
10
), p.
100908
.10.1115/1.4007390
63.
Jiang
,
Z.
,
Apley
,
D. W.
, and
Chen
,
W.
, January
2015
, “
Surrogate Preposterior Analyses for Predicting and Enhancing Identifiability in Model Calibration
,”
Int. J. Uncertainty Quantif.
,
5
(
4
), pp.
341
359
.10.1615/Int.J.UncertaintyQuantification.2015012627
64.
Arendt
,
P. D.
,
Apley
,
D. W.
,
Chen
,
W.
,
Lamb
,
D.
, and
Gorsich
,
D.
,
2012
, “
Improving Identifiability in Model Calibration Using Multiple Responses
,”
ASME J. Mech. Des.
,
134
(
10
), p.
100909
.10.1115/1.4007573
You do not currently have access to this content.