Predicting hospital admissions from individual patient data (IPD): an applied example to explore key elements driving external validity
Objective To explore factors that potentially impact external validation performance while developing and validating a prognostic model for hospital admissions (HAs) in complex older general practice patients.Study design and setting Using individual participant data from four cluster-randomised tri...
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMJ Publishing Group
2021-08-01
|
| Series: | BMJ Open |
| Online Access: | https://bmjopen.bmj.com/content/11/8/e045572.full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850138119252213760 |
|---|---|
| author | Rafael Perera Marjan Van Den Akker Kym I E Snell Joerg J Meerpohl Ferdinand M Gerlach Daniela Küllenberg de Gaudry Ana Isabel González-González Jeanet Blom Christiane Muth Walter Emil Haefeli Karin M A Swart Petra Elders Henrik Rudolf Ulrich Thiem Andreas Daniel Meid Truc Sophia Dinh Donna Bosch-Lenders Hans J Trampisch Benno Flaig Ghainsom Kom |
| author_facet | Rafael Perera Marjan Van Den Akker Kym I E Snell Joerg J Meerpohl Ferdinand M Gerlach Daniela Küllenberg de Gaudry Ana Isabel González-González Jeanet Blom Christiane Muth Walter Emil Haefeli Karin M A Swart Petra Elders Henrik Rudolf Ulrich Thiem Andreas Daniel Meid Truc Sophia Dinh Donna Bosch-Lenders Hans J Trampisch Benno Flaig Ghainsom Kom |
| author_sort | Rafael Perera |
| collection | DOAJ |
| description | Objective To explore factors that potentially impact external validation performance while developing and validating a prognostic model for hospital admissions (HAs) in complex older general practice patients.Study design and setting Using individual participant data from four cluster-randomised trials conducted in the Netherlands and Germany, we used logistic regression to develop a prognostic model to predict all-cause HAs within a 6-month follow-up period. A stratified intercept was used to account for heterogeneity in baseline risk between the studies. The model was validated both internally and by using internal-external cross-validation (IECV).Results Prior HAs, physical components of the health-related quality of life comorbidity index, and medication-related variables were used in the final model. While achieving moderate discriminatory performance, internal bootstrap validation revealed a pronounced risk of overfitting. The results of the IECV, in which calibration was highly variable even after accounting for between-study heterogeneity, agreed with this finding. Heterogeneity was equally reflected in differing baseline risk, predictor effects and absolute risk predictions.Conclusions Predictor effect heterogeneity and differing baseline risk can explain the limited external performance of HA prediction models. With such drivers known, model adjustments in external validation settings (eg, intercept recalibration, complete updating) can be applied more purposefully.Trial registration number PROSPERO id: CRD42018088129. |
| format | Article |
| id | doaj-art-0014ac625b614f349b8b0d9ca196fab9 |
| institution | OA Journals |
| issn | 2044-6055 |
| language | English |
| publishDate | 2021-08-01 |
| publisher | BMJ Publishing Group |
| record_format | Article |
| series | BMJ Open |
| spelling | doaj-art-0014ac625b614f349b8b0d9ca196fab92025-08-20T02:30:39ZengBMJ Publishing GroupBMJ Open2044-60552021-08-0111810.1136/bmjopen-2020-045572Predicting hospital admissions from individual patient data (IPD): an applied example to explore key elements driving external validityRafael Perera0Marjan Van Den Akker1Kym I E Snell2Joerg J Meerpohl3Ferdinand M Gerlach4Daniela Küllenberg de Gaudry5Ana Isabel González-González6Jeanet Blom7Christiane Muth8Walter Emil Haefeli9Karin M A Swart10Petra Elders11Henrik Rudolf12Ulrich Thiem13Andreas Daniel Meid14Truc Sophia Dinh15Donna Bosch-Lenders16Hans J Trampisch17Benno Flaig18Ghainsom Kom19Department of Primary Care Health Sciences, University of Oxford, Oxford, UK21 Institute of General Practice, University of Frankfurt, Frankfurt am Main, GermanyInstitute of Applied Health Research, University of Birmingham, Birmingham, UKInstitute for Evidence in Medicine, Medical Center – Faculty of Medicine, University of Freiburg, Freiburg im Breisgau, GermanyInstitute of General Practice, Goethe University Frankfurt, Frankfurt am Main, Germanyresearcher4Network for Research on Chronicity, Primary Care, and Health Promotion (RICAPPS), Tenerife, Spain5 Department of Public Health and Primary Care, Leiden University Medical Center, Leiden, Zuid-Holland, Netherlands20 Department of General Practice and Family Medicine, University Hospital OWL of Bielefeld University Campus Hospital Lippe, Detmold, Germany1 Department of Clinical Pharmacology and Pharmacoepidemiology, University of Heidelberg, Heidelberg, Baden-Württemberg, GermanyPHARMO Institute for Drug Outcomes Research, Utrecht, NetherlandsHealth Behaviours & Chronic Diseases, Amsterdam Public Health Research Institute, Amsterdam, The NetherlandsMedical Informatics, Biometry and Epidemiology, Ruhr-University Bochum, Bochum, GermanyChair of Geriatrics and Gerontology, University Clinic Eppendorf, Hamburg, GermanyDepartment of Clinical Pharmacology & Pharmacoepidemiology, Heidelberg University, Heidelberg, Baden-Württemberg, GermanyInstitute of General Practice, Goethe University Frankfurt, Frankfurt am Main, Hessen, GermanySchool of CAPHRI, Department of Family Medicine, Maastricht University, Maastricht, The NetherlandsDepartment of Medical Informatics, Biometry and Epidemiology, Ruhr University, Bochum, GermanyInstitute of General Practice, Goethe University Frankfurt, Frankfurt am Main, GermanyTechniker Krankenkasse (TK), Hamburg, GermanyObjective To explore factors that potentially impact external validation performance while developing and validating a prognostic model for hospital admissions (HAs) in complex older general practice patients.Study design and setting Using individual participant data from four cluster-randomised trials conducted in the Netherlands and Germany, we used logistic regression to develop a prognostic model to predict all-cause HAs within a 6-month follow-up period. A stratified intercept was used to account for heterogeneity in baseline risk between the studies. The model was validated both internally and by using internal-external cross-validation (IECV).Results Prior HAs, physical components of the health-related quality of life comorbidity index, and medication-related variables were used in the final model. While achieving moderate discriminatory performance, internal bootstrap validation revealed a pronounced risk of overfitting. The results of the IECV, in which calibration was highly variable even after accounting for between-study heterogeneity, agreed with this finding. Heterogeneity was equally reflected in differing baseline risk, predictor effects and absolute risk predictions.Conclusions Predictor effect heterogeneity and differing baseline risk can explain the limited external performance of HA prediction models. With such drivers known, model adjustments in external validation settings (eg, intercept recalibration, complete updating) can be applied more purposefully.Trial registration number PROSPERO id: CRD42018088129.https://bmjopen.bmj.com/content/11/8/e045572.full |
| spellingShingle | Rafael Perera Marjan Van Den Akker Kym I E Snell Joerg J Meerpohl Ferdinand M Gerlach Daniela Küllenberg de Gaudry Ana Isabel González-González Jeanet Blom Christiane Muth Walter Emil Haefeli Karin M A Swart Petra Elders Henrik Rudolf Ulrich Thiem Andreas Daniel Meid Truc Sophia Dinh Donna Bosch-Lenders Hans J Trampisch Benno Flaig Ghainsom Kom Predicting hospital admissions from individual patient data (IPD): an applied example to explore key elements driving external validity BMJ Open |
| title | Predicting hospital admissions from individual patient data (IPD): an applied example to explore key elements driving external validity |
| title_full | Predicting hospital admissions from individual patient data (IPD): an applied example to explore key elements driving external validity |
| title_fullStr | Predicting hospital admissions from individual patient data (IPD): an applied example to explore key elements driving external validity |
| title_full_unstemmed | Predicting hospital admissions from individual patient data (IPD): an applied example to explore key elements driving external validity |
| title_short | Predicting hospital admissions from individual patient data (IPD): an applied example to explore key elements driving external validity |
| title_sort | predicting hospital admissions from individual patient data ipd an applied example to explore key elements driving external validity |
| url | https://bmjopen.bmj.com/content/11/8/e045572.full |
| work_keys_str_mv | AT rafaelperera predictinghospitaladmissionsfromindividualpatientdataipdanappliedexampletoexplorekeyelementsdrivingexternalvalidity AT marjanvandenakker predictinghospitaladmissionsfromindividualpatientdataipdanappliedexampletoexplorekeyelementsdrivingexternalvalidity AT kymiesnell predictinghospitaladmissionsfromindividualpatientdataipdanappliedexampletoexplorekeyelementsdrivingexternalvalidity AT joergjmeerpohl predictinghospitaladmissionsfromindividualpatientdataipdanappliedexampletoexplorekeyelementsdrivingexternalvalidity AT ferdinandmgerlach predictinghospitaladmissionsfromindividualpatientdataipdanappliedexampletoexplorekeyelementsdrivingexternalvalidity AT danielakullenbergdegaudry predictinghospitaladmissionsfromindividualpatientdataipdanappliedexampletoexplorekeyelementsdrivingexternalvalidity AT anaisabelgonzalezgonzalez predictinghospitaladmissionsfromindividualpatientdataipdanappliedexampletoexplorekeyelementsdrivingexternalvalidity AT jeanetblom predictinghospitaladmissionsfromindividualpatientdataipdanappliedexampletoexplorekeyelementsdrivingexternalvalidity AT christianemuth predictinghospitaladmissionsfromindividualpatientdataipdanappliedexampletoexplorekeyelementsdrivingexternalvalidity AT walteremilhaefeli predictinghospitaladmissionsfromindividualpatientdataipdanappliedexampletoexplorekeyelementsdrivingexternalvalidity AT karinmaswart predictinghospitaladmissionsfromindividualpatientdataipdanappliedexampletoexplorekeyelementsdrivingexternalvalidity AT petraelders predictinghospitaladmissionsfromindividualpatientdataipdanappliedexampletoexplorekeyelementsdrivingexternalvalidity AT henrikrudolf predictinghospitaladmissionsfromindividualpatientdataipdanappliedexampletoexplorekeyelementsdrivingexternalvalidity AT ulrichthiem predictinghospitaladmissionsfromindividualpatientdataipdanappliedexampletoexplorekeyelementsdrivingexternalvalidity AT andreasdanielmeid predictinghospitaladmissionsfromindividualpatientdataipdanappliedexampletoexplorekeyelementsdrivingexternalvalidity AT trucsophiadinh predictinghospitaladmissionsfromindividualpatientdataipdanappliedexampletoexplorekeyelementsdrivingexternalvalidity AT donnaboschlenders predictinghospitaladmissionsfromindividualpatientdataipdanappliedexampletoexplorekeyelementsdrivingexternalvalidity AT hansjtrampisch predictinghospitaladmissionsfromindividualpatientdataipdanappliedexampletoexplorekeyelementsdrivingexternalvalidity AT bennoflaig predictinghospitaladmissionsfromindividualpatientdataipdanappliedexampletoexplorekeyelementsdrivingexternalvalidity AT ghainsomkom predictinghospitaladmissionsfromindividualpatientdataipdanappliedexampletoexplorekeyelementsdrivingexternalvalidity |