Systematic review finds that study data not published in full text articles have unclear impact on meta-analyses results in medical research.
<h4>Background</h4>A meta-analysis as part of a systematic review aims to provide a thorough, comprehensive and unbiased statistical summary of data from the literature. However, relevant study results could be missing from a meta-analysis because of selective publication and inadequate...
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2017-01-01
|
| Series: | PLoS ONE |
| Online Access: | https://doi.org/10.1371/journal.pone.0176210 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849682962656788480 |
|---|---|
| author | Christine M Schmucker Anette Blümle Lisa K Schell Guido Schwarzer Patrick Oeller Laura Cabrera Erik von Elm Matthias Briel Joerg J Meerpohl OPEN consortium |
| author_facet | Christine M Schmucker Anette Blümle Lisa K Schell Guido Schwarzer Patrick Oeller Laura Cabrera Erik von Elm Matthias Briel Joerg J Meerpohl OPEN consortium |
| author_sort | Christine M Schmucker |
| collection | DOAJ |
| description | <h4>Background</h4>A meta-analysis as part of a systematic review aims to provide a thorough, comprehensive and unbiased statistical summary of data from the literature. However, relevant study results could be missing from a meta-analysis because of selective publication and inadequate dissemination. If missing outcome data differ systematically from published ones, a meta-analysis will be biased with an inaccurate assessment of the intervention effect. As part of the EU-funded OPEN project (www.open-project.eu) we conducted a systematic review that assessed whether the inclusion of data that were not published at all and/or published only in the grey literature influences pooled effect estimates in meta-analyses and leads to different interpretation.<h4>Methods and findings</h4>Systematic review of published literature (methodological research projects). Four bibliographic databases were searched up to February 2016 without restriction of publication year or language. Methodological research projects were considered eligible for inclusion if they reviewed a cohort of meta-analyses which (i) compared pooled effect estimates of meta-analyses of health care interventions according to publication status of data or (ii) examined whether the inclusion of unpublished or grey literature data impacts the result of a meta-analysis. Seven methodological research projects including 187 meta-analyses comparing pooled treatment effect estimates according to different publication status were identified. Two research projects showed that published data showed larger pooled treatment effects in favour of the intervention than unpublished or grey literature data (Ratio of ORs 1.15, 95% CI 1.04-1.28 and 1.34, 95% CI 1.09-1.66). In the remaining research projects pooled effect estimates and/or overall findings were not significantly changed by the inclusion of unpublished and/or grey literature data. The precision of the pooled estimate was increased with narrower 95% confidence interval.<h4>Conclusions</h4>Although we may anticipate that systematic reviews and meta-analyses not including unpublished or grey literature study results are likely to overestimate the treatment effects, current empirical research shows that this is only the case in a minority of reviews. Therefore, currently, a meta-analyst should particularly consider time, effort and costs when adding such data to their analysis. Future research is needed to identify which reviews may benefit most from including unpublished or grey data. |
| format | Article |
| id | doaj-art-a9ed76b1457446e48e69b82ccb2f9b3e |
| institution | DOAJ |
| issn | 1932-6203 |
| language | English |
| publishDate | 2017-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj-art-a9ed76b1457446e48e69b82ccb2f9b3e2025-08-20T03:24:02ZengPublic Library of Science (PLoS)PLoS ONE1932-62032017-01-01124e017621010.1371/journal.pone.0176210Systematic review finds that study data not published in full text articles have unclear impact on meta-analyses results in medical research.Christine M SchmuckerAnette BlümleLisa K SchellGuido SchwarzerPatrick OellerLaura CabreraErik von ElmMatthias BrielJoerg J MeerpohlOPEN consortium<h4>Background</h4>A meta-analysis as part of a systematic review aims to provide a thorough, comprehensive and unbiased statistical summary of data from the literature. However, relevant study results could be missing from a meta-analysis because of selective publication and inadequate dissemination. If missing outcome data differ systematically from published ones, a meta-analysis will be biased with an inaccurate assessment of the intervention effect. As part of the EU-funded OPEN project (www.open-project.eu) we conducted a systematic review that assessed whether the inclusion of data that were not published at all and/or published only in the grey literature influences pooled effect estimates in meta-analyses and leads to different interpretation.<h4>Methods and findings</h4>Systematic review of published literature (methodological research projects). Four bibliographic databases were searched up to February 2016 without restriction of publication year or language. Methodological research projects were considered eligible for inclusion if they reviewed a cohort of meta-analyses which (i) compared pooled effect estimates of meta-analyses of health care interventions according to publication status of data or (ii) examined whether the inclusion of unpublished or grey literature data impacts the result of a meta-analysis. Seven methodological research projects including 187 meta-analyses comparing pooled treatment effect estimates according to different publication status were identified. Two research projects showed that published data showed larger pooled treatment effects in favour of the intervention than unpublished or grey literature data (Ratio of ORs 1.15, 95% CI 1.04-1.28 and 1.34, 95% CI 1.09-1.66). In the remaining research projects pooled effect estimates and/or overall findings were not significantly changed by the inclusion of unpublished and/or grey literature data. The precision of the pooled estimate was increased with narrower 95% confidence interval.<h4>Conclusions</h4>Although we may anticipate that systematic reviews and meta-analyses not including unpublished or grey literature study results are likely to overestimate the treatment effects, current empirical research shows that this is only the case in a minority of reviews. Therefore, currently, a meta-analyst should particularly consider time, effort and costs when adding such data to their analysis. Future research is needed to identify which reviews may benefit most from including unpublished or grey data.https://doi.org/10.1371/journal.pone.0176210 |
| spellingShingle | Christine M Schmucker Anette Blümle Lisa K Schell Guido Schwarzer Patrick Oeller Laura Cabrera Erik von Elm Matthias Briel Joerg J Meerpohl OPEN consortium Systematic review finds that study data not published in full text articles have unclear impact on meta-analyses results in medical research. PLoS ONE |
| title | Systematic review finds that study data not published in full text articles have unclear impact on meta-analyses results in medical research. |
| title_full | Systematic review finds that study data not published in full text articles have unclear impact on meta-analyses results in medical research. |
| title_fullStr | Systematic review finds that study data not published in full text articles have unclear impact on meta-analyses results in medical research. |
| title_full_unstemmed | Systematic review finds that study data not published in full text articles have unclear impact on meta-analyses results in medical research. |
| title_short | Systematic review finds that study data not published in full text articles have unclear impact on meta-analyses results in medical research. |
| title_sort | systematic review finds that study data not published in full text articles have unclear impact on meta analyses results in medical research |
| url | https://doi.org/10.1371/journal.pone.0176210 |
| work_keys_str_mv | AT christinemschmucker systematicreviewfindsthatstudydatanotpublishedinfulltextarticleshaveunclearimpactonmetaanalysesresultsinmedicalresearch AT anetteblumle systematicreviewfindsthatstudydatanotpublishedinfulltextarticleshaveunclearimpactonmetaanalysesresultsinmedicalresearch AT lisakschell systematicreviewfindsthatstudydatanotpublishedinfulltextarticleshaveunclearimpactonmetaanalysesresultsinmedicalresearch AT guidoschwarzer systematicreviewfindsthatstudydatanotpublishedinfulltextarticleshaveunclearimpactonmetaanalysesresultsinmedicalresearch AT patrickoeller systematicreviewfindsthatstudydatanotpublishedinfulltextarticleshaveunclearimpactonmetaanalysesresultsinmedicalresearch AT lauracabrera systematicreviewfindsthatstudydatanotpublishedinfulltextarticleshaveunclearimpactonmetaanalysesresultsinmedicalresearch AT erikvonelm systematicreviewfindsthatstudydatanotpublishedinfulltextarticleshaveunclearimpactonmetaanalysesresultsinmedicalresearch AT matthiasbriel systematicreviewfindsthatstudydatanotpublishedinfulltextarticleshaveunclearimpactonmetaanalysesresultsinmedicalresearch AT joergjmeerpohl systematicreviewfindsthatstudydatanotpublishedinfulltextarticleshaveunclearimpactonmetaanalysesresultsinmedicalresearch AT openconsortium systematicreviewfindsthatstudydatanotpublishedinfulltextarticleshaveunclearimpactonmetaanalysesresultsinmedicalresearch |