Project example 9: Meta-modelling

Project type	Methodological study
Methods	Meta-modelling Orinary least squares Generalized least squares Spatial interpolation Gaussian processes
Deliverables	Report
Publications	Comparison of four meta-models for uncertainty analysis. Lieuw On MML, Al M, Heeg B. Masterthesis for the Institute of Health Policy and Management, Erasmus University Rotterdam.

Probabilistic sensitivity analysis involves multiple runs of cost-effectiveness models with randomly sampled values of the model inputs. This can be a time-intensive process, especially in the case of micro-simulation models. Meta-modelling approaches can be applied as an alternative to reduce computation time, but the meta-model must be accurate enough to replace the individual simulation siwht the original model. In this project four different meta-models were designed and compared in terms of computation time and accuracy: Ordinalyre Leas Squares (OLS), Generalized Least Squares (GLS), Spatial Interpolations (SI), and Glaussian Processes (GP). OLS and GLS are linear regression methods, whereas SI and GP are complex interpolation techniques. GP is the only methods that can incorporate prior information, which could lead to better results.

An example is presented in the figure above. The blue dots represent the actual values of probabilistic analyses with the original model. The pink line is the result of the linear regression. This represents the best result OLS can give, but it is rather poor compared to the results of the Gaussian process meta-model in red. Interpolation techniques (SI and GP) use a correction function that "pulls" the line towards the supplied data points.

The computation time of all meta-models was well below the time needed to perform probabilistic sensitivity analyses with the original model. The results showed that SI and GP were the preferred meta-models. The compuations time needed to perform SI and GP was slightly longer than for OLS and GLS, but the prediction quality was higher. The methods and results of this self-funded project were documented in an internal report, and have been published as a graduation thesis.