ML-AMPSIT: Machine Learning-based Automated Multi-method Parameter Sensitivity and Importance analysis Tool
<p>The accurate calibration of parameters in atmospheric and Earth system models is crucial for improving their performance but remains a challenge due to their inherent complexity, which is reflected in input–output relationships often characterised by multiple interactions between the parame...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Copernicus Publications
2025-01-01
|
| Series: | Geoscientific Model Development |
| Online Access: | https://gmd.copernicus.org/articles/18/433/2025/gmd-18-433-2025.pdf |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832584917696905216 |
|---|---|
| author | D. Di Santo C. He F. Chen L. Giovannini |
| author_facet | D. Di Santo C. He F. Chen L. Giovannini |
| author_sort | D. Di Santo |
| collection | DOAJ |
| description | <p>The accurate calibration of parameters in atmospheric and Earth system models is crucial for improving their performance but remains a challenge due to their inherent complexity, which is reflected in input–output relationships often characterised by multiple interactions between the parameters, thus hindering the use of simple sensitivity analysis methods. This paper introduces the Machine Learning-based Automated Multi-method Parameter Sensitivity and Importance analysis Tool (ML-AMPSIT), a new tool designed with the aim of providing a simple and flexible framework to estimate the sensitivity and importance of parameters in complex numerical weather prediction models. This tool leverages the strengths of multiple regression-based and probabilistic machine learning methods, including LASSO (see the list of abbreviations in Appendix B), support vector machine, classification and regression trees, random forest, extreme gradient boosting, Gaussian process regression, and Bayesian ridge regression. These regression algorithms are used to construct computationally inexpensive surrogate models to effectively predict the impact of input parameter variations on model output, thereby significantly reducing the computational burden of running high-fidelity models for sensitivity analysis. Moreover, the multi-method approach allows for a comparative analysis of the results. Through a detailed case study with the Weather Research and Forecasting (WRF) model coupled with the Noah-MP land surface model, ML-AMPSIT is demonstrated to efficiently predict the effects of varying the values of Noah-MP model parameters with a relatively small number of model runs by simulating a sea breeze circulation over an idealised flat domain. This paper points out how ML-AMPSIT can be an efficient tool for performing sensitivity and importance analysis for complex models, guiding the user through the different steps and allowing for a simplification and automatisation of the process.</p> |
| format | Article |
| id | doaj-art-2bb35c0e25134935b8b0ea420ff99c8f |
| institution | Kabale University |
| issn | 1991-959X 1991-9603 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Copernicus Publications |
| record_format | Article |
| series | Geoscientific Model Development |
| spelling | doaj-art-2bb35c0e25134935b8b0ea420ff99c8f2025-01-27T07:58:20ZengCopernicus PublicationsGeoscientific Model Development1991-959X1991-96032025-01-011843345910.5194/gmd-18-433-2025ML-AMPSIT: Machine Learning-based Automated Multi-method Parameter Sensitivity and Importance analysis ToolD. Di Santo0C. He1F. Chen2L. Giovannini3Department of Civil, Environmental and Mechanical Engineering, University of Trento, Trento, ItalyNSF National Center for Atmospheric Research (NCAR), Boulder, CO, USADivision of Environment and Sustainability, Hong Kong University of Science and Technology, Hong Kong SAR, ChinaDepartment of Civil, Environmental and Mechanical Engineering, University of Trento, Trento, Italy<p>The accurate calibration of parameters in atmospheric and Earth system models is crucial for improving their performance but remains a challenge due to their inherent complexity, which is reflected in input–output relationships often characterised by multiple interactions between the parameters, thus hindering the use of simple sensitivity analysis methods. This paper introduces the Machine Learning-based Automated Multi-method Parameter Sensitivity and Importance analysis Tool (ML-AMPSIT), a new tool designed with the aim of providing a simple and flexible framework to estimate the sensitivity and importance of parameters in complex numerical weather prediction models. This tool leverages the strengths of multiple regression-based and probabilistic machine learning methods, including LASSO (see the list of abbreviations in Appendix B), support vector machine, classification and regression trees, random forest, extreme gradient boosting, Gaussian process regression, and Bayesian ridge regression. These regression algorithms are used to construct computationally inexpensive surrogate models to effectively predict the impact of input parameter variations on model output, thereby significantly reducing the computational burden of running high-fidelity models for sensitivity analysis. Moreover, the multi-method approach allows for a comparative analysis of the results. Through a detailed case study with the Weather Research and Forecasting (WRF) model coupled with the Noah-MP land surface model, ML-AMPSIT is demonstrated to efficiently predict the effects of varying the values of Noah-MP model parameters with a relatively small number of model runs by simulating a sea breeze circulation over an idealised flat domain. This paper points out how ML-AMPSIT can be an efficient tool for performing sensitivity and importance analysis for complex models, guiding the user through the different steps and allowing for a simplification and automatisation of the process.</p>https://gmd.copernicus.org/articles/18/433/2025/gmd-18-433-2025.pdf |
| spellingShingle | D. Di Santo C. He F. Chen L. Giovannini ML-AMPSIT: Machine Learning-based Automated Multi-method Parameter Sensitivity and Importance analysis Tool Geoscientific Model Development |
| title | ML-AMPSIT: Machine Learning-based Automated Multi-method Parameter Sensitivity and Importance analysis Tool |
| title_full | ML-AMPSIT: Machine Learning-based Automated Multi-method Parameter Sensitivity and Importance analysis Tool |
| title_fullStr | ML-AMPSIT: Machine Learning-based Automated Multi-method Parameter Sensitivity and Importance analysis Tool |
| title_full_unstemmed | ML-AMPSIT: Machine Learning-based Automated Multi-method Parameter Sensitivity and Importance analysis Tool |
| title_short | ML-AMPSIT: Machine Learning-based Automated Multi-method Parameter Sensitivity and Importance analysis Tool |
| title_sort | ml ampsit machine learning based automated multi method parameter sensitivity and importance analysis tool |
| url | https://gmd.copernicus.org/articles/18/433/2025/gmd-18-433-2025.pdf |
| work_keys_str_mv | AT ddisanto mlampsitmachinelearningbasedautomatedmultimethodparametersensitivityandimportanceanalysistool AT che mlampsitmachinelearningbasedautomatedmultimethodparametersensitivityandimportanceanalysistool AT fchen mlampsitmachinelearningbasedautomatedmultimethodparametersensitivityandimportanceanalysistool AT lgiovannini mlampsitmachinelearningbasedautomatedmultimethodparametersensitivityandimportanceanalysistool |