Bayesian history matching

Bayesian history matching is a statistical method for calibrating complex computer models. The equations inside many scientific computer models contain parameters which have a true value, but that true value is often unknown; history matching is one technique for learning what these parameters could be.

The name originates from the oil industry, where it refers to any technique for making sure oil reservoir models match up with historical oil production records.[1] Since then, history matching has been widely used in many areas of science and engineering, including galaxy formation,[2] disease modelling,[3] climate science,[4] and traffic simulation.[5]

The basis of history matching is to use observed data to rule-out any parameter settings which are ``implausible’’. Since computer models are often too slow to individually check every possible parameter setting, this is usually done with the help of an emulator. For a set of potential parameter settings ${\displaystyle {\boldsymbol {\theta }}}$, their implausibility ${\displaystyle I({\boldsymbol {\theta }})}$ can be calculated as:

${\displaystyle I({\boldsymbol {\theta }})={\frac {|E[f({\boldsymbol {\theta }})]-y|}{\sqrt {Var[f({\boldsymbol {\theta }})]}}}}$

where ${\displaystyle E[f({\boldsymbol {\theta }})]}$ is the expected output of the computer model for that parameter setting, and ${\displaystyle Var[f({\boldsymbol {\theta }})]}$ represents the uncertainties around the computer model output for that parameter setting. In other words, a parameter setting is scored based on how different the computer model output is to the real world observations, relative to how much uncertainty there is.

For computer models that output only one value, an implausibility of 3 is considered a good threshold for rejecting parameter settings.[6] For computer models which output more than one output, other thresholds can be used.[7] A key component of history matching is the notion of iterative refocussing,[8] where new computer model simulations can be chosen to better improve the emulator and the calibration, based on preliminary results.

References

1. Craig, Peter S.; Goldstein, Michael; Seheult, Allan H.; Smith, James A. (1997). "Pressure Matching for Hydrocarbon Reservoirs: A Case Study in the Use of Bayes Linear Strategies for Large Computer Experiments". In Gatsonis, Constantine; Hodges, James S.; Kass, Robert E.; McCulloch, Robert; Rossi, Peter; Singpurwalla, Nozer D. (eds.). Case Studies in Bayesian Statistics. Lecture Notes in Statistics. Vol. 121. New York, NY: Springer. pp. 37–93. doi:10.1007/978-1-4612-2290-3_2. ISBN 978-1-4612-2290-3.
2. Vernon, Ian; Goldstein, Michael; Bower, Richard (February 1, 2014). "Galaxy Formation: Bayesian History Matching for the Observable Universe". Statistical Science. 29 (1): 81–90. arXiv:1405.4976. doi:10.1214/12-STS412. S2CID 18315892 – via Project Euclid.
3. Andrianakis, Ioannis; Vernon, Ian R.; McCreesh, Nicky; McKinley, Trevelyan J.; Oakley, Jeremy E.; Nsubuga, Rebecca N.; Goldstein, Michael; White, Richard G. (January 1, 2015). "Bayesian history matching of complex infectious disease models using emulation: a tutorial and a case study on HIV in Uganda". PLOS Computational Biology. 11 (1): e1003968. Bibcode:2015PLSCB..11E3968A. doi:10.1371/journal.pcbi.1003968. PMC 4288726. PMID 25569850.
4. Williamson, Daniel; Goldstein, Michael; Allison, Lesley; Blaker, Adam; Challenor, Peter; Jackson, Laura; Yamazaki, Kuniko (October 1, 2013). "History matching for exploring and reducing climate model parameter space using observations and a large perturbed physics ensemble". Climate Dynamics. 41 (7): 1703–1729. Bibcode:2013ClDy...41.1703W. doi:10.1007/s00382-013-1896-4. S2CID 120737289.
5. Boukouvalas, Alexis; Sykes, Pete; Cornford, Dan; Maruri-Aguilar, Hugo (June 1, 2014). "Bayesian Precalibration of a Large Stochastic Microsimulation Model" (PDF). IEEE Transactions on Intelligent Transportation Systems. 15 (3): 1337–1347. doi:10.1109/TITS.2014.2304394. S2CID 16209605.
6. Pukelsheim, Friedrich (May 1, 1994). "The Three Sigma Rule" (PDF). The American Statistician. 48 (2): 88–91. doi:10.1080/00031305.1994.10476030. S2CID 122587510.
7. Vernon, Ian; Goldstein, Michael; Bower, Richard G. (December 1, 2010). "Galaxy formation: a Bayesian uncertainty analysis". Bayesian Analysis. 5 (4): 619–669. doi:10.1214/10-BA524 – via Project Euclid.
8. Salter, James M.; Williamson, Daniel B.; Scinocca, John; Kharin, Viatcheslav (October 2, 2019). "Uncertainty Quantification for Computer Models With Spatial Output Using Calibration-Optimal Bases". Journal of the American Statistical Association. 114 (528): 1800–1814. doi:10.1080/01621459.2018.1514306. hdl:10871/33707.