Systematic review of the empirical evidence of study publication bias and outcome reporting bias.
<h4>Background</h4>The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias has been recognised as a potential threat to the val...
Saved in:
| Main Authors: | , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2008-08-01
|
| Series: | PLoS ONE |
| Online Access: | https://doi.org/10.1371/journal.pone.0003081 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850166342642040832 |
|---|---|
| author | Kerry Dwan Douglas G Altman Juan A Arnaiz Jill Bloom An-Wen Chan Eugenia Cronin Evelyne Decullier Philippa J Easterbrook Erik Von Elm Carrol Gamble Davina Ghersi John P A Ioannidis John Simes Paula R Williamson |
| author_facet | Kerry Dwan Douglas G Altman Juan A Arnaiz Jill Bloom An-Wen Chan Eugenia Cronin Evelyne Decullier Philippa J Easterbrook Erik Von Elm Carrol Gamble Davina Ghersi John P A Ioannidis John Simes Paula R Williamson |
| author_sort | Kerry Dwan |
| collection | DOAJ |
| description | <h4>Background</h4>The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias has been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. Until recently, outcome reporting bias has received less attention.<h4>Methodology/principal findings</h4>We review and summarise the evidence from a series of cohort studies that have assessed study publication bias and outcome reporting bias in randomised controlled trials. Sixteen studies were eligible of which only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Eleven of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies.<h4>Conclusions</h4>Recent work provides direct empirical evidence for the existence of study publication bias and outcome reporting bias. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials. |
| format | Article |
| id | doaj-art-e2e30c0c8b0e4c799c930e4dfbc0c5b0 |
| institution | OA Journals |
| issn | 1932-6203 |
| language | English |
| publishDate | 2008-08-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj-art-e2e30c0c8b0e4c799c930e4dfbc0c5b02025-08-20T02:21:29ZengPublic Library of Science (PLoS)PLoS ONE1932-62032008-08-0138e308110.1371/journal.pone.0003081Systematic review of the empirical evidence of study publication bias and outcome reporting bias.Kerry DwanDouglas G AltmanJuan A ArnaizJill BloomAn-Wen ChanEugenia CroninEvelyne DecullierPhilippa J EasterbrookErik Von ElmCarrol GambleDavina GhersiJohn P A IoannidisJohn SimesPaula R Williamson<h4>Background</h4>The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias has been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. Until recently, outcome reporting bias has received less attention.<h4>Methodology/principal findings</h4>We review and summarise the evidence from a series of cohort studies that have assessed study publication bias and outcome reporting bias in randomised controlled trials. Sixteen studies were eligible of which only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Eleven of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies.<h4>Conclusions</h4>Recent work provides direct empirical evidence for the existence of study publication bias and outcome reporting bias. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.https://doi.org/10.1371/journal.pone.0003081 |
| spellingShingle | Kerry Dwan Douglas G Altman Juan A Arnaiz Jill Bloom An-Wen Chan Eugenia Cronin Evelyne Decullier Philippa J Easterbrook Erik Von Elm Carrol Gamble Davina Ghersi John P A Ioannidis John Simes Paula R Williamson Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS ONE |
| title | Systematic review of the empirical evidence of study publication bias and outcome reporting bias. |
| title_full | Systematic review of the empirical evidence of study publication bias and outcome reporting bias. |
| title_fullStr | Systematic review of the empirical evidence of study publication bias and outcome reporting bias. |
| title_full_unstemmed | Systematic review of the empirical evidence of study publication bias and outcome reporting bias. |
| title_short | Systematic review of the empirical evidence of study publication bias and outcome reporting bias. |
| title_sort | systematic review of the empirical evidence of study publication bias and outcome reporting bias |
| url | https://doi.org/10.1371/journal.pone.0003081 |
| work_keys_str_mv | AT kerrydwan systematicreviewoftheempiricalevidenceofstudypublicationbiasandoutcomereportingbias AT douglasgaltman systematicreviewoftheempiricalevidenceofstudypublicationbiasandoutcomereportingbias AT juanaarnaiz systematicreviewoftheempiricalevidenceofstudypublicationbiasandoutcomereportingbias AT jillbloom systematicreviewoftheempiricalevidenceofstudypublicationbiasandoutcomereportingbias AT anwenchan systematicreviewoftheempiricalevidenceofstudypublicationbiasandoutcomereportingbias AT eugeniacronin systematicreviewoftheempiricalevidenceofstudypublicationbiasandoutcomereportingbias AT evelynedecullier systematicreviewoftheempiricalevidenceofstudypublicationbiasandoutcomereportingbias AT philippajeasterbrook systematicreviewoftheempiricalevidenceofstudypublicationbiasandoutcomereportingbias AT erikvonelm systematicreviewoftheempiricalevidenceofstudypublicationbiasandoutcomereportingbias AT carrolgamble systematicreviewoftheempiricalevidenceofstudypublicationbiasandoutcomereportingbias AT davinaghersi systematicreviewoftheempiricalevidenceofstudypublicationbiasandoutcomereportingbias AT johnpaioannidis systematicreviewoftheempiricalevidenceofstudypublicationbiasandoutcomereportingbias AT johnsimes systematicreviewoftheempiricalevidenceofstudypublicationbiasandoutcomereportingbias AT paularwilliamson systematicreviewoftheempiricalevidenceofstudypublicationbiasandoutcomereportingbias |