Decomposition of skill scores for conditional verification: impact of Atlantic Multidecadal Oscillation phases on the predictability of decadal temperature forecasts
<p>As the performance of weather and climate forecasting systems and their benchmark systems are generally not homogeneous in time and space and may vary in specific situations, improvements in certain situations or subsets have different effects on overall skill. We present a decomposition of...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Copernicus Publications
2025-01-01
|
| Series: | Geoscientific Model Development |
| Online Access: | https://gmd.copernicus.org/articles/18/361/2025/gmd-18-361-2025.pdf |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832591477564243968 |
|---|---|
| author | A. Richling J. Grieger H. W. Rust |
| author_facet | A. Richling J. Grieger H. W. Rust |
| author_sort | A. Richling |
| collection | DOAJ |
| description | <p>As the performance of weather and climate forecasting systems and their benchmark systems are generally not homogeneous in time and space and may vary in specific situations, improvements in certain situations or subsets have different effects on overall skill. We present a decomposition of skill scores for the conditional verification of such systems. The aim is to evaluate the performance of a system individually for predefined subsets with respect to the overall performance. The overall skill score is decomposed into a weighted sum representing <i>subset contributions</i>, where each individual contribution is the product of the following: (1) the <i>subset skill score</i>, assessing the performance of a forecast system compared to a reference system for a particular subset; (2) the <i>frequency weighting</i>, accounting for varying subset size; and (3) the <i>reference weighting</i>, relating the performance of the reference system in the individual subsets to the performance of the full data set. The decomposition and its interpretation are exemplified using synthetic data. Subsequently, we use it for a practical example from the field of decadal climate prediction: an evaluation of the Atlantic European near-surface temperature forecast from the German “Mittelfristige Klimaprognosen” (MiKlip) initiative decadal prediction system that is conditional on different Atlantic Multidecadal Oscillation (AMO) phases during initialization. With respect to the chosen western European North Atlantic sector, the decadal prediction system “preop-dcpp-HR” performs better than the uninitialized simulations mostly due to contributions during the positive AMO phase driven by the subset skill score. Compared to the low-resolution system (preop-LR), no overall performance benefits are made in this region, but positive contributions are achieved for initialization in neutral AMO phases. Additionally, the decomposition reveals a strong imbalance among the subsets (defined by AMO phases) in terms of reference weighting, allowing for insightful interpretation and conclusions. This skill score decomposition framework for conditional verification is a valuable tool to analyze the effect of physical processes on forecast performance and, consequently, supports model development and the improvement of operational forecasts.</p> |
| format | Article |
| id | doaj-art-d698aafa7b9e4f0cb3318e4b97ef92f2 |
| institution | Kabale University |
| issn | 1991-959X 1991-9603 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Copernicus Publications |
| record_format | Article |
| series | Geoscientific Model Development |
| spelling | doaj-art-d698aafa7b9e4f0cb3318e4b97ef92f22025-01-22T11:52:38ZengCopernicus PublicationsGeoscientific Model Development1991-959X1991-96032025-01-011836137510.5194/gmd-18-361-2025Decomposition of skill scores for conditional verification: impact of Atlantic Multidecadal Oscillation phases on the predictability of decadal temperature forecastsA. Richling0J. Grieger1H. W. Rust2Institute of Meteorology, Freie Universität Berlin, Carl-Heinrich-Becker Weg 6–10, 12165 Berlin, GermanyInstitute of Meteorology, Freie Universität Berlin, Carl-Heinrich-Becker Weg 6–10, 12165 Berlin, GermanyInstitute of Meteorology, Freie Universität Berlin, Carl-Heinrich-Becker Weg 6–10, 12165 Berlin, Germany<p>As the performance of weather and climate forecasting systems and their benchmark systems are generally not homogeneous in time and space and may vary in specific situations, improvements in certain situations or subsets have different effects on overall skill. We present a decomposition of skill scores for the conditional verification of such systems. The aim is to evaluate the performance of a system individually for predefined subsets with respect to the overall performance. The overall skill score is decomposed into a weighted sum representing <i>subset contributions</i>, where each individual contribution is the product of the following: (1) the <i>subset skill score</i>, assessing the performance of a forecast system compared to a reference system for a particular subset; (2) the <i>frequency weighting</i>, accounting for varying subset size; and (3) the <i>reference weighting</i>, relating the performance of the reference system in the individual subsets to the performance of the full data set. The decomposition and its interpretation are exemplified using synthetic data. Subsequently, we use it for a practical example from the field of decadal climate prediction: an evaluation of the Atlantic European near-surface temperature forecast from the German “Mittelfristige Klimaprognosen” (MiKlip) initiative decadal prediction system that is conditional on different Atlantic Multidecadal Oscillation (AMO) phases during initialization. With respect to the chosen western European North Atlantic sector, the decadal prediction system “preop-dcpp-HR” performs better than the uninitialized simulations mostly due to contributions during the positive AMO phase driven by the subset skill score. Compared to the low-resolution system (preop-LR), no overall performance benefits are made in this region, but positive contributions are achieved for initialization in neutral AMO phases. Additionally, the decomposition reveals a strong imbalance among the subsets (defined by AMO phases) in terms of reference weighting, allowing for insightful interpretation and conclusions. This skill score decomposition framework for conditional verification is a valuable tool to analyze the effect of physical processes on forecast performance and, consequently, supports model development and the improvement of operational forecasts.</p>https://gmd.copernicus.org/articles/18/361/2025/gmd-18-361-2025.pdf |
| spellingShingle | A. Richling J. Grieger H. W. Rust Decomposition of skill scores for conditional verification: impact of Atlantic Multidecadal Oscillation phases on the predictability of decadal temperature forecasts Geoscientific Model Development |
| title | Decomposition of skill scores for conditional verification: impact of Atlantic Multidecadal Oscillation phases on the predictability of decadal temperature forecasts |
| title_full | Decomposition of skill scores for conditional verification: impact of Atlantic Multidecadal Oscillation phases on the predictability of decadal temperature forecasts |
| title_fullStr | Decomposition of skill scores for conditional verification: impact of Atlantic Multidecadal Oscillation phases on the predictability of decadal temperature forecasts |
| title_full_unstemmed | Decomposition of skill scores for conditional verification: impact of Atlantic Multidecadal Oscillation phases on the predictability of decadal temperature forecasts |
| title_short | Decomposition of skill scores for conditional verification: impact of Atlantic Multidecadal Oscillation phases on the predictability of decadal temperature forecasts |
| title_sort | decomposition of skill scores for conditional verification impact of atlantic multidecadal oscillation phases on the predictability of decadal temperature forecasts |
| url | https://gmd.copernicus.org/articles/18/361/2025/gmd-18-361-2025.pdf |
| work_keys_str_mv | AT arichling decompositionofskillscoresforconditionalverificationimpactofatlanticmultidecadaloscillationphasesonthepredictabilityofdecadaltemperatureforecasts AT jgrieger decompositionofskillscoresforconditionalverificationimpactofatlanticmultidecadaloscillationphasesonthepredictabilityofdecadaltemperatureforecasts AT hwrust decompositionofskillscoresforconditionalverificationimpactofatlanticmultidecadaloscillationphasesonthepredictabilityofdecadaltemperatureforecasts |