Evidence Communication Rules for Policy (ECR-P) critical appraisal tool
Abstract Background Scientific papers increasingly put forward scientific-based policy recommendations (SPRs) as a means of closing the circle of science, policy and practice. Assessing the quality of such SPRs is crucial, especially within the context of a systematic review. Here, we present ECR-P...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMC
2025-01-01
|
| Series: | Systematic Reviews |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s13643-025-02757-8 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832594954675814400 |
|---|---|
| author | Evangelos Danopoulos John A. D. Aston Aarushi Shah Claudia R. Schneider |
| author_facet | Evangelos Danopoulos John A. D. Aston Aarushi Shah Claudia R. Schneider |
| author_sort | Evangelos Danopoulos |
| collection | DOAJ |
| description | Abstract Background Scientific papers increasingly put forward scientific-based policy recommendations (SPRs) as a means of closing the circle of science, policy and practice. Assessing the quality of such SPRs is crucial, especially within the context of a systematic review. Here, we present ECR-P (Evidence Communication Rules for Policy)—a critical appraisal tool that we have developed, which can be used in assessing not only the quality of SPRs but also the quality of their evidence base and how effectively these have both been communicated. Methods The rationale behind ECR-P centres on three dimensions of quality; two are the well-established concepts of internal and external validity. Here, we introduce a third—evidence communication—encompassing both evidence veracity and quality of communication. Elements of the three dimensions of quality are considered within the context of the five rules of evidence communication. These are as follows: inform, not persuade; offer balance, not false balance; disclose uncertainties; state evidence quality and pre-empt misunderstandings. Results Development of ECR-P has been carried out by an interdisciplinary team and was piloted with a systematic review reported more fully elsewhere. ECR-P comprises a set of preliminary considerations which capture key aspects for the assessment, leading on to the main tool whose structure is domain-based, each domain mapping to one of the five rules of evidence communication. The domains include 25 signalling questions designed to obtain essential information for the critical appraisal. The questions focus on either the study’s evidence or the policy recommendations. Domain-based judgement is derived from responses to the signalling questions and an accompanying algorithm, followed by an overall quality judgement. Conclusions ECR-P has been designed to provide a standardised and transparent approach to assess the quality and communication of SPRs and their evidence base. The tool, which could be applied across all scientific fields, has been developed to fit primarily with the systematic reviewing process but could also serve as a stand-alone tool. Besides review assessors, it can also be used by policymakers, researchers, peer reviewers, editors and any other stakeholders interested in evidence-based policymaking and high-quality evidence communication. We encourage further independent testing of the tool in real-world evidence-based research. |
| format | Article |
| id | doaj-art-19329bf3cbce4098884efb3bb8ea1599 |
| institution | Kabale University |
| issn | 2046-4053 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | BMC |
| record_format | Article |
| series | Systematic Reviews |
| spelling | doaj-art-19329bf3cbce4098884efb3bb8ea15992025-01-19T12:12:00ZengBMCSystematic Reviews2046-40532025-01-011411910.1186/s13643-025-02757-8Evidence Communication Rules for Policy (ECR-P) critical appraisal toolEvangelos Danopoulos0John A. D. Aston1Aarushi Shah2Claudia R. Schneider3Statistical Laboratory, Department of Pure Mathematics and Mathematical Statistics, University of CambridgeStatistical Laboratory, Department of Pure Mathematics and Mathematical Statistics, University of CambridgeStatistical Laboratory, Department of Pure Mathematics and Mathematical Statistics, University of CambridgeStatistical Laboratory, Department of Pure Mathematics and Mathematical Statistics, University of CambridgeAbstract Background Scientific papers increasingly put forward scientific-based policy recommendations (SPRs) as a means of closing the circle of science, policy and practice. Assessing the quality of such SPRs is crucial, especially within the context of a systematic review. Here, we present ECR-P (Evidence Communication Rules for Policy)—a critical appraisal tool that we have developed, which can be used in assessing not only the quality of SPRs but also the quality of their evidence base and how effectively these have both been communicated. Methods The rationale behind ECR-P centres on three dimensions of quality; two are the well-established concepts of internal and external validity. Here, we introduce a third—evidence communication—encompassing both evidence veracity and quality of communication. Elements of the three dimensions of quality are considered within the context of the five rules of evidence communication. These are as follows: inform, not persuade; offer balance, not false balance; disclose uncertainties; state evidence quality and pre-empt misunderstandings. Results Development of ECR-P has been carried out by an interdisciplinary team and was piloted with a systematic review reported more fully elsewhere. ECR-P comprises a set of preliminary considerations which capture key aspects for the assessment, leading on to the main tool whose structure is domain-based, each domain mapping to one of the five rules of evidence communication. The domains include 25 signalling questions designed to obtain essential information for the critical appraisal. The questions focus on either the study’s evidence or the policy recommendations. Domain-based judgement is derived from responses to the signalling questions and an accompanying algorithm, followed by an overall quality judgement. Conclusions ECR-P has been designed to provide a standardised and transparent approach to assess the quality and communication of SPRs and their evidence base. The tool, which could be applied across all scientific fields, has been developed to fit primarily with the systematic reviewing process but could also serve as a stand-alone tool. Besides review assessors, it can also be used by policymakers, researchers, peer reviewers, editors and any other stakeholders interested in evidence-based policymaking and high-quality evidence communication. We encourage further independent testing of the tool in real-world evidence-based research.https://doi.org/10.1186/s13643-025-02757-8Critical appraisalQuality assessmentRisk of bias toolPolicy recommendationsPolicymaking |
| spellingShingle | Evangelos Danopoulos John A. D. Aston Aarushi Shah Claudia R. Schneider Evidence Communication Rules for Policy (ECR-P) critical appraisal tool Systematic Reviews Critical appraisal Quality assessment Risk of bias tool Policy recommendations Policymaking |
| title | Evidence Communication Rules for Policy (ECR-P) critical appraisal tool |
| title_full | Evidence Communication Rules for Policy (ECR-P) critical appraisal tool |
| title_fullStr | Evidence Communication Rules for Policy (ECR-P) critical appraisal tool |
| title_full_unstemmed | Evidence Communication Rules for Policy (ECR-P) critical appraisal tool |
| title_short | Evidence Communication Rules for Policy (ECR-P) critical appraisal tool |
| title_sort | evidence communication rules for policy ecr p critical appraisal tool |
| topic | Critical appraisal Quality assessment Risk of bias tool Policy recommendations Policymaking |
| url | https://doi.org/10.1186/s13643-025-02757-8 |
| work_keys_str_mv | AT evangelosdanopoulos evidencecommunicationrulesforpolicyecrpcriticalappraisaltool AT johnadaston evidencecommunicationrulesforpolicyecrpcriticalappraisaltool AT aarushishah evidencecommunicationrulesforpolicyecrpcriticalappraisaltool AT claudiarschneider evidencecommunicationrulesforpolicyecrpcriticalappraisaltool |