Simulation-Based Inference

Last update: 07 Dec 2024 23:38
First version: 19 September 2024

i.e., how to do statistical inference when calculating the probability of a data set under a model is intractable, but simulating the model is straightforward. I got tired of shoe-horning all my references on this into the notebook on one particular way of doing this that I used to work on, so have a stub. I rather like the literature review in my 2021 "matching random features" preprint, if you really want to know what I think about the field, at least as of then...

Mark C. Beaumont, "Approximation Bayesian Computation in Evolution and Ecology", Annual Review of Ecology and Systematics 41 (2010): 379--406 [Thanks to Prof, Beaumont for kindly sharing a preprint]
Kyle Cranmer, Johann Brehmer, and Gilles Louppe, "The frontier of simulation-based inference", Proceedings of the National Academy of Sciences (USA) 117 (2020): 30055--30062, arxiv:1911.01429 [I find it a bit remarkable that this paper completely ignored indirect inference, simulated moments, etc., but it's good on what it does cover]
Christian Gouriéroux and Alain Monfort, Simulation-Based Econometric Methods

Niccolò Dalmasso, Rafael Izbicki, Ann B. Lee, "Confidence Sets and Hypothesis Testing in a Likelihood-Free Inference Setting", arxiv:2002.10399
Luca Masserano, Tommaso Dorigo, Rafael Izbicki, Mikael Kuusela, Ann B. Lee, "Simulator-Based Inference with Waldo: Confidence Regions by Leveraging Prediction Algorithms and Posterior Estimators for Inverse Problems", arxiv:2205.15680
X. Z. Tang, E. R. Tracy, A. D. Boozer, A. deBrauw, and R. Brown, "Symbol sequence statistics in noisy chaotic signal reconstruction", Physical Review E 51 (1995): 3871
F. V. Tkachov [Comments]
- "Approaching the Parameter Estimation Quality of Maximum Likelihood via Generalized Moments", arxiv:physics/0001019
- "Quasi-optimal observables: Attaining the quality of maximal likelihood in parameter estimation when only a MC event generator is available," arxiv:physics/0108030
Simon N. Wood, "Statistical inference for noisy nonlinear ecological dynamic systems", Nature 466 (1102--1104)

Chris P. Barnes, Sarah Filippi, Michael P.H. Stumpf, Thomas Thorne, "Considerate Approaches to Achieving Sufficiency for ABC model selection", Statistics and Computing 22 (2012): 1181--1197, arxiv:1106.6281 [Given a large candidate of summary statistics, this uses an information-theoretic characterization of sufficiency to efficiently search for a subset of statistics which is approximately sufficient.]
M. G. B. Blum, M. A. Nunes, D. Prangle, and S. A. Sisson, "A Comparative Review of Dimension Reduction Methods in Approximate Bayesian Computation", Statistical Science 28 (2013): 189--208 [A lot of this carries over to II]
Paul Fearnhead, Dennis Prangle, "Constructing summary statistics for approximate Bayesian computation: semi-automatic approximate Bayesian computation", Journal of the Royal Statistical Society B 74 (2012): 419--474 [If we knew the posterior means of the parameters as functions of the data, those functions would make excellent summary statistics. Of course the whole point of ABC is to find the posterior distribution, so this seems like a vicious circle. They offer a clever way to break the circle by beginning with bad summary statistics and then trying to learn the posterior expectation functions on the basis of the initial approximation to the posterior distribution. Not circular, but not exactly easy.]
Jean-Jacques Forneron, Serena Ng, "The ABC of Simulation Estimation with Auxiliary Statistics", arxiv:1501.01265
David T. Frazier, Gael M. Martin, Christian P. Robert and Judith Rousseau, "Asymptotic Properties of Approximate Bayesian Computation", Biometrika 105 (2018): 593--607, arxiv:1607.06903
J.-M. Marin, N. Pillai, C. P. Robert, J. Rousseau, "Relevant statistics for Bayesian model choice", arxiv:1110.4700
Dennis Prangle, Paul Fearnhead, Murray P. Cox, Patrick J. Biggs, Nigel P. French, "Semi-automatic selection of summary statistics for ABC model choice", arxiv:1302.5624
Michael Vespe, "The potential of likelihood-free inference of cosmological parameters with weak lensing data", Proceedings of the International Astronomical Union 10:S306 (2015): 90--93

CRS, "A Note on Simulation-Based Inference by Matching Random Features", arxiv:2111.09220

Rainier Barrett, Mehrad Ansari, Gourab Ghoshal, Andrew D. White, "Simulation-Based Inference with Approximately Correct Parameters via Maximum Entropy", arxiv:2104.09668
Johanna Bertl, Gregory Ewing, Carolin Kosiol, Andreas Futschik, "Approximate Maximum Likelihood Estimation", arxiv:1507.04553
Michael G. B. Blum, "Approximate Bayesian Computation: A Nonparametric Perspective", Journal of the American Statistical Association 105 (2010): 1178--1187, arxiv:0904.0635
Carles Bretó, Daihai He, Edward L. Ionides, Aaron A. King, "Time series analysis via mechanistic models", Annals of Applied Statistics 3 (2009): 319--348, arxiv:0802.0021
Francois-Xavier Briol, Alessandro Barp, Andrew B. Duncan, Mark Girolami, "Statistical Inference for Generative Models with Maximum Mean Discrepancy", arxiv:1906.05944
Jukka Corander, Ulpu Remes and Timo Koski, "Likelihood-free Model Choice for Simulator-based Models with the Jensen--Shannon Divergence", arxiv:2206.04110
Maximilian Dax, Jonas Wildberger, Simon Buchholz, Stephen R. Green, Jakob H. Macke, Bernhard Schölkopf, "Flow Matching for Scalable Simulation-Based Inference", arxiv:2305.17161
Pierre Del Moral, Arnaud Doucet and Ajay Jasra, "An Adaptive Sequential Monte Carlo Method for Approximate Bayesian Computation" [PDF preprint]
Christopher Drovandi, David T Frazier, "A Comparison of Likelihood-Free Methods With and Without Summary Statistics", arxiv:2103.02407
Christopher C. Drovandi, Anthony N. Pettitt, Malcolm J. Faddy, "Approximate Bayesian computation using indirect inference", Journal of the Royal Statistical Society C 60 (2011): 317--337
David T. Frazier, Christopher Drovandi, David J. Nott, "Bayesian Synthetic Likelihood", arxiv:2305.05120
Thomas Gaskin, Grigorios A. Pavliotis, Mark Girolami, "Neural parameter calibration for large-scale multi-agent models", arxiv:2209.13565
Patrik Róbert Gerber, Yury Polyanskiy, "Likelihood-free hypothesis testing", arxiv:2211.01126
Mark Girolami, Anne-Marie Lyne, Heiko Strathmann, Daniel Simpson, Yves Atchade, "Playing Russian Roulette with Intractable Likelihoods", arxiv:1306.4032
Aude Grelaud, Christian Robert, Jean-Michel Marin, Francois Rodolphe, Jean-Francois Taly, "ABC likelihood-freee methods for model choice in Gibbs random fields", arxiv:0807.2767
Lukas Heinrich, Siddharth Mishra-Sharma, Chris Pollard, Philipp Windischhofer, "Hierarchical Neural Simulation-Based Inference Over Event Ensembles", arxiv:2306.12584
Joeri Hermans, Arnaud Delaunoy, FranÃ§ois Rozet, Antoine Wehenkel, Volodimir Begy, Gilles Louppe, "A Trust Crisis In Simulation-Based Inference? Your Posterior Approximations Can Be Unfaithful", arxiv:2110.06581 [TMLR]
Ying Hung, Li-Hsiang Lin and C. F. Jeff Wu, "Optimal Simulator Selection", Journal of the American Statistical Association 118 (2023): 1264--1271
Ajay Jasra, Nikolas Kantas, Elena Ehrlich, "Approximate Inference for Observation Driven Time Series Models with Intractable Likelihoods", arxiv:1303.7318
Robert Mariano, Til Schuermann and Melyvn J. Weeks (eds.), Simulation-Baed Inference in Econometrics: Methods and Applications
Jean-Michel Marin, Pierre Pudlo, Christian P. Robert, Robin Ryder, "Approximate Bayesian Computational methods", arxiv:1101.0955
Lorenzo Pacchiardi, Ritabrata Dutta, "Score Matched Neural Exponential Families for Likelihood-Free Inference", arxiv:2012.10903
Umberto Picchini, "Inference for SDE models via Approximate Bayesian Computation", arxiv:1204.5459
Gyanendra Pokharel, Rob Deardon, "Emulation-based inference for spatial infectious disease transmission models incorporating event time uncertainty", Scandanavian Journal of Statistics <49 (2022): 4565--479
Oliver Ratmann, Anton Camacho, Adam Meijer, Gé Donker, "Statistical modelling of summary values leads to accurate Approximate Bayesian Computations", arxiv:1305.4283 [Sounds a bit like what Wood does in his Nature paper]
Oliver Ratmann, Pierre Pudlo, Sylvia Richardson, Christian Robert, "Monte Carlo algorithms for model assessment via conflicting summaries", arxiv:1106.5919
F. J. Rubio, Adam M. Johansen, "A Simple Approach to Maximum Intractable Likelihood Estimation", Electronic Journal of Statistics 7 (2013): 1632--1654, arxiv:1301.0463
Mikaela Sundberg, "The dynamics of coordinated comparisons: How simulationists in astrophysics, oceanography and meteorology create standards for results", Social Studies of Science 41 (2011): 107--125
Suzanne Thornton, Wentao Li, Minge Xie, "Approximate confidence distribution computing", arxiv:2206.01707
Rui Tuo, Shiyuan He, Arash Pourhabib, Yu Ding and Jianhua Z. Huang, "A Reproducing Kernel Hilbert Space Approach to Functional Calibration of Computer Models", Journal of the American Statistical Association 118 (2023): 883--897
Cheng Wang, Carter T. Butts, John Hipp, and Cynthia M. Lakon, "Model Adequacy Checking/Goodness-of-fit Testing for Behavior in Joint Dynamic Network/Behavior Models, with an Extension to Two-mode Networks", Sociological Methods and Research 51 (2022): 1886--1919
Yan Wang, Xiaowei Yue, Rui Tuo, Jeffrey H. Hunt, Jianjun Shi, "Effective model calibration via sensible variable identification and adjustment with application to composite fuselage simulation", Annals of Applied Statistics 14 (2020): 1759--1776