Prediction accuracy of discrete choice experiments in health-related research: a systematic review and meta-analysisResearch in context
Summary: Background: Discrete choice experiments (DCEs) are increasingly used to inform the design of health products and services. It is essential to understand the extent to which DCEs provide reliable predictions outside of experimental settings in real-world decision-making situations. We aimed...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025-01-01
|
| Series: | EClinicalMedicine |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2589537024005443 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832591807564742656 |
|---|---|
| author | Ying Zhang Thi Quynh Anh Ho Fern Terris-Prestholt Matthew Quaife Esther de Bekker-Grob Peter Vickerman Jason J. Ong |
| author_facet | Ying Zhang Thi Quynh Anh Ho Fern Terris-Prestholt Matthew Quaife Esther de Bekker-Grob Peter Vickerman Jason J. Ong |
| author_sort | Ying Zhang |
| collection | DOAJ |
| description | Summary: Background: Discrete choice experiments (DCEs) are increasingly used to inform the design of health products and services. It is essential to understand the extent to which DCEs provide reliable predictions outside of experimental settings in real-world decision-making situations. We aimed to compare the prediction accuracy of stated preferences with real-world choices, as modelled from DCE data. Methods: We searched six databases for health-related studies that used DCE to assess external validity and reported on predicted versus real-world choices, up to July 2024. A generalised linear mixed model was used for a meta-analysis to jointly pool the sensitivity and specificity. Heterogeneity was assessed using the I2 statistic, and sources of heterogeneity using meta-regression. This study is registered with PROSPERO (CRD42023451545). Findings: We identified 14 relevant studies, of which 10 were included in the meta-analysis. Most studies were conducted in high-income countries (11/14, 79%) from the European region (9/14, 64%) and analysed using mixed logit models (5/14, 36%). Pooled sensitivity and specificity estimates were 89% (95% CI:77–95, I2 = 97%) and 52% (95% CI:32–72, I2 = 95%), respectively. The area under the SROC curve (AUC) was 0.81 (95% CI:0.77–0.84). Our meta-regression found that DCEs for prevention-related choices had higher sensitivity than treatment-related choices. DCEs conducted under clinical settings and analysed using the heteroskedastic multinomial logit model, incorporating systematic preference heterogeneity and random opt-out utility, had higher specificity than non-clinical settings and alternative models. Interpretation: DCEs are valuable for capturing health-related preferences and possess reasonable external validity to predict health-related behaviours, particularly for opt-in choices. Contextual factors (e.g., type of intervention, study setting, analysis method) influenced the predictive accuracy. Funding: JJO is supported by an Australian National Health and Medical Research Council Emerging Leadership Investigator Grant (GNT1193955). EBG is supported by the Dutch Research Council (NWO-Talent-Scheme-Vidi-Grant No, 09150171910002). YZ is supported by an Australian Government Research Training Program (RTP) scholarship. |
| format | Article |
| id | doaj-art-ecb1609849074926b162d9aef0d88abd |
| institution | Kabale University |
| issn | 2589-5370 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Elsevier |
| record_format | Article |
| series | EClinicalMedicine |
| spelling | doaj-art-ecb1609849074926b162d9aef0d88abd2025-01-22T05:43:10ZengElsevierEClinicalMedicine2589-53702025-01-0179102965Prediction accuracy of discrete choice experiments in health-related research: a systematic review and meta-analysisResearch in contextYing Zhang0Thi Quynh Anh Ho1Fern Terris-Prestholt2Matthew Quaife3Esther de Bekker-Grob4Peter Vickerman5Jason J. Ong6School of Translational Medicine, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Victoria, Australia; Melbourne Sexual Health Centre, Alfred Health, Melbourne, Victoria, AustraliaDeakin Health Economics, Institute for Health Transformation, School of Health and Social Development, Faculty of Health, Deakin University, Melbourne, Victoria, AustraliaWarwick Medical School, University of Warwick, Coventry, United KingdomPatient Centered Research, Evidera, London, United KingdomErasmus School of Health Policy & Management, Erasmus University, Rotterdam, the Netherlands; Erasmus Choice Modelling Centre, Erasmus University, Rotterdam, the Netherlands; Erasmus Centre for Health Economics Rotterdam, Erasmus University, Rotterdam, the NetherlandsBristol Medical School, University of Bristol, Bristol, United KingdomSchool of Translational Medicine, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Victoria, Australia; Melbourne Sexual Health Centre, Alfred Health, Melbourne, Victoria, Australia; London School of Hygiene and Tropical Medicine, London, United Kingdom; Corresponding author. Melbourne Sexual Health Centre, 580 Swanston Street, Carlton, Victoria 3053, Australia.Summary: Background: Discrete choice experiments (DCEs) are increasingly used to inform the design of health products and services. It is essential to understand the extent to which DCEs provide reliable predictions outside of experimental settings in real-world decision-making situations. We aimed to compare the prediction accuracy of stated preferences with real-world choices, as modelled from DCE data. Methods: We searched six databases for health-related studies that used DCE to assess external validity and reported on predicted versus real-world choices, up to July 2024. A generalised linear mixed model was used for a meta-analysis to jointly pool the sensitivity and specificity. Heterogeneity was assessed using the I2 statistic, and sources of heterogeneity using meta-regression. This study is registered with PROSPERO (CRD42023451545). Findings: We identified 14 relevant studies, of which 10 were included in the meta-analysis. Most studies were conducted in high-income countries (11/14, 79%) from the European region (9/14, 64%) and analysed using mixed logit models (5/14, 36%). Pooled sensitivity and specificity estimates were 89% (95% CI:77–95, I2 = 97%) and 52% (95% CI:32–72, I2 = 95%), respectively. The area under the SROC curve (AUC) was 0.81 (95% CI:0.77–0.84). Our meta-regression found that DCEs for prevention-related choices had higher sensitivity than treatment-related choices. DCEs conducted under clinical settings and analysed using the heteroskedastic multinomial logit model, incorporating systematic preference heterogeneity and random opt-out utility, had higher specificity than non-clinical settings and alternative models. Interpretation: DCEs are valuable for capturing health-related preferences and possess reasonable external validity to predict health-related behaviours, particularly for opt-in choices. Contextual factors (e.g., type of intervention, study setting, analysis method) influenced the predictive accuracy. Funding: JJO is supported by an Australian National Health and Medical Research Council Emerging Leadership Investigator Grant (GNT1193955). EBG is supported by the Dutch Research Council (NWO-Talent-Scheme-Vidi-Grant No, 09150171910002). YZ is supported by an Australian Government Research Training Program (RTP) scholarship.http://www.sciencedirect.com/science/article/pii/S2589537024005443Discrete choice experimentExternal validityPredictive accuracyStated preferencesRevealed preferences |
| spellingShingle | Ying Zhang Thi Quynh Anh Ho Fern Terris-Prestholt Matthew Quaife Esther de Bekker-Grob Peter Vickerman Jason J. Ong Prediction accuracy of discrete choice experiments in health-related research: a systematic review and meta-analysisResearch in context EClinicalMedicine Discrete choice experiment External validity Predictive accuracy Stated preferences Revealed preferences |
| title | Prediction accuracy of discrete choice experiments in health-related research: a systematic review and meta-analysisResearch in context |
| title_full | Prediction accuracy of discrete choice experiments in health-related research: a systematic review and meta-analysisResearch in context |
| title_fullStr | Prediction accuracy of discrete choice experiments in health-related research: a systematic review and meta-analysisResearch in context |
| title_full_unstemmed | Prediction accuracy of discrete choice experiments in health-related research: a systematic review and meta-analysisResearch in context |
| title_short | Prediction accuracy of discrete choice experiments in health-related research: a systematic review and meta-analysisResearch in context |
| title_sort | prediction accuracy of discrete choice experiments in health related research a systematic review and meta analysisresearch in context |
| topic | Discrete choice experiment External validity Predictive accuracy Stated preferences Revealed preferences |
| url | http://www.sciencedirect.com/science/article/pii/S2589537024005443 |
| work_keys_str_mv | AT yingzhang predictionaccuracyofdiscretechoiceexperimentsinhealthrelatedresearchasystematicreviewandmetaanalysisresearchincontext AT thiquynhanhho predictionaccuracyofdiscretechoiceexperimentsinhealthrelatedresearchasystematicreviewandmetaanalysisresearchincontext AT fernterrisprestholt predictionaccuracyofdiscretechoiceexperimentsinhealthrelatedresearchasystematicreviewandmetaanalysisresearchincontext AT matthewquaife predictionaccuracyofdiscretechoiceexperimentsinhealthrelatedresearchasystematicreviewandmetaanalysisresearchincontext AT estherdebekkergrob predictionaccuracyofdiscretechoiceexperimentsinhealthrelatedresearchasystematicreviewandmetaanalysisresearchincontext AT petervickerman predictionaccuracyofdiscretechoiceexperimentsinhealthrelatedresearchasystematicreviewandmetaanalysisresearchincontext AT jasonjong predictionaccuracyofdiscretechoiceexperimentsinhealthrelatedresearchasystematicreviewandmetaanalysisresearchincontext |