Multiverse Analysis for Dynamic Network Models: Investigating the Influence of Plausible Alternative Modeling Choices
Specifying complex time series models typically allows for a wide range of plausible analysis strategies. However, researchers typically perform and report only a single, preferred analysis while ignoring alternatives that could yield different conclusions. As a remedy, we propose multiverse analysi...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
PsychOpen GOLD/ Leibniz Institute for Psychology
2025-06-01
|
| Series: | Methodology |
| Subjects: | |
| Online Access: | https://doi.org/10.5964/meth.15665 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849408290770911232 |
|---|---|
| author | Björn S. Siepe Daniel W. Heck |
| author_facet | Björn S. Siepe Daniel W. Heck |
| author_sort | Björn S. Siepe |
| collection | DOAJ |
| description | Specifying complex time series models typically allows for a wide range of plausible analysis strategies. However, researchers typically perform and report only a single, preferred analysis while ignoring alternatives that could yield different conclusions. As a remedy, we propose multiverse analysis to investigate the robustness of dynamic network analysis to arbitrary modeling choices. We focus on group iterative multiple model estimation (GIMME), a highly data-driven approach, and re-analyze two datasets (combined n = 199). We vary seven modeling parameters, resulting in 3,888 fitted models. Group-level and to a lesser extent subgroup-level results were mostly stable. Individual-level estimates were more heterogeneous, with some decisions strongly influencing results and conclusions. The robustness of GIMME to alternative modeling choices depends on the level of analysis. For some individuals, results may differ strongly even when changing the algorithm only slightly. Multiverse analysis is a valuable tool for checking the robustness of results from time series models. |
| format | Article |
| id | doaj-art-3f4ef214d2154d80be3c3f4e003af6cd |
| institution | Kabale University |
| issn | 1614-2241 |
| language | English |
| publishDate | 2025-06-01 |
| publisher | PsychOpen GOLD/ Leibniz Institute for Psychology |
| record_format | Article |
| series | Methodology |
| spelling | doaj-art-3f4ef214d2154d80be3c3f4e003af6cd2025-08-20T03:35:50ZengPsychOpen GOLD/ Leibniz Institute for PsychologyMethodology1614-22412025-06-0121212314310.5964/meth.15665meth.15665Multiverse Analysis for Dynamic Network Models: Investigating the Influence of Plausible Alternative Modeling ChoicesBjörn S. Siepe0https://orcid.org/0000-0002-9558-4648Daniel W. Heck1https://orcid.org/0000-0002-6302-9252Psychological Methods Lab, Department of Psychology, University of Marburg, Marburg, GermanyPsychological Methods Lab, Department of Psychology, University of Marburg, Marburg, GermanySpecifying complex time series models typically allows for a wide range of plausible analysis strategies. However, researchers typically perform and report only a single, preferred analysis while ignoring alternatives that could yield different conclusions. As a remedy, we propose multiverse analysis to investigate the robustness of dynamic network analysis to arbitrary modeling choices. We focus on group iterative multiple model estimation (GIMME), a highly data-driven approach, and re-analyze two datasets (combined n = 199). We vary seven modeling parameters, resulting in 3,888 fitted models. Group-level and to a lesser extent subgroup-level results were mostly stable. Individual-level estimates were more heterogeneous, with some decisions strongly influencing results and conclusions. The robustness of GIMME to alternative modeling choices depends on the level of analysis. For some individuals, results may differ strongly even when changing the algorithm only slightly. Multiverse analysis is a valuable tool for checking the robustness of results from time series models.https://doi.org/10.5964/meth.15665time seriesnetwork analysismultiverseheterogeneitygimme |
| spellingShingle | Björn S. Siepe Daniel W. Heck Multiverse Analysis for Dynamic Network Models: Investigating the Influence of Plausible Alternative Modeling Choices Methodology time series network analysis multiverse heterogeneity gimme |
| title | Multiverse Analysis for Dynamic Network Models: Investigating the Influence of Plausible Alternative Modeling Choices |
| title_full | Multiverse Analysis for Dynamic Network Models: Investigating the Influence of Plausible Alternative Modeling Choices |
| title_fullStr | Multiverse Analysis for Dynamic Network Models: Investigating the Influence of Plausible Alternative Modeling Choices |
| title_full_unstemmed | Multiverse Analysis for Dynamic Network Models: Investigating the Influence of Plausible Alternative Modeling Choices |
| title_short | Multiverse Analysis for Dynamic Network Models: Investigating the Influence of Plausible Alternative Modeling Choices |
| title_sort | multiverse analysis for dynamic network models investigating the influence of plausible alternative modeling choices |
| topic | time series network analysis multiverse heterogeneity gimme |
| url | https://doi.org/10.5964/meth.15665 |
| work_keys_str_mv | AT bjornssiepe multiverseanalysisfordynamicnetworkmodelsinvestigatingtheinfluenceofplausiblealternativemodelingchoices AT danielwheck multiverseanalysisfordynamicnetworkmodelsinvestigatingtheinfluenceofplausiblealternativemodelingchoices |