Use of computerised adaptive testing to reduce the number of items in patient-reported hip and knee outcome scores: an analysis of the NHS England National Patient-Reported Outcome Measures programme
Objective Over 160 000 participants per year complete the 12-item Oxford Hip and Knee Scores (OHS/OKS) as part of the NHS England Patient-Reported Outcome Measures (PROMs) programme. We used a modern computational approach, known as computerised adaptive testing (CAT), to simulate individually tailo...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMJ Publishing Group
2022-07-01
|
| Series: | BMJ Open |
| Online Access: | https://bmjopen.bmj.com/content/12/7/e059415.full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832576064103120896 |
|---|---|
| author | Jose Maria Valderas Andrew D Toms Jonathan Peter Evans Christopher Gibbons |
| author_facet | Jose Maria Valderas Andrew D Toms Jonathan Peter Evans Christopher Gibbons |
| author_sort | Jose Maria Valderas |
| collection | DOAJ |
| description | Objective Over 160 000 participants per year complete the 12-item Oxford Hip and Knee Scores (OHS/OKS) as part of the NHS England Patient-Reported Outcome Measures (PROMs) programme. We used a modern computational approach, known as computerised adaptive testing (CAT), to simulate individually tailored OHS and OKS assessment, with the goal of reducing the number of questions a patient must complete without compromising measurement accuracy.Methods We fit the 2018/2019 PROMs data to an item response theory (IRT) model. We assessed IRT model assumptions alongside reliability. We used parameters from the IRT model with data from 2017/2018 to simulate CAT assessments. Two simulations were run until a prespecified SE of measurement was met (SE=0.32 and SE=0.45). We compared the number of questions required to meet each cut-off and assessed the correlation between the full-length and CAT administration.Results We conducted IRT analysis using 40 432 OHS and 44 714 OKS observations. The OHS and OKS were both unidimensional (root mean square error of approximation 0.08 and 0.07, respectively) and marginal reliability 0.91 and 0.90. The CAT, with a precision limit of SE=0.32 and SE=0.45, required a median of four items (IQR 1) and two items (IQR 1), respectively, for the OHS, and median of four items (IQR 2) and two items (IQR 0) for the OKS. This represents a potential 82% reduction in PROM length. In the context of 160 000 yearly assessments, these methodologies could result in the omission of some 1 280 000 redundant questions per year, which equates to 40 000 hours of patient time.Conclusion The application of IRT to the OHS and OKS produces an efficient and substantially reduced CAT. We have demonstrated a path to reduce the burden and potentially increase the compliance for these ubiquitous outcome measures without compromising measurement accuracy. |
| format | Article |
| id | doaj-art-a0b809b62a4e4fbfb871c74c02eb41dd |
| institution | Kabale University |
| issn | 2044-6055 |
| language | English |
| publishDate | 2022-07-01 |
| publisher | BMJ Publishing Group |
| record_format | Article |
| series | BMJ Open |
| spelling | doaj-art-a0b809b62a4e4fbfb871c74c02eb41dd2025-01-31T13:10:11ZengBMJ Publishing GroupBMJ Open2044-60552022-07-0112710.1136/bmjopen-2021-059415Use of computerised adaptive testing to reduce the number of items in patient-reported hip and knee outcome scores: an analysis of the NHS England National Patient-Reported Outcome Measures programmeJose Maria Valderas0Andrew D Toms1Jonathan Peter Evans2Christopher Gibbons3Health Services and Policy Research Group, University of Exeter Medical School, Exeter, UKRoyal Devon and Exeter NHS Foundation Trust, Exeter, UKHealth Services and Policy Research Group, University of Exeter Medical School, Exeter, UKSection of Patient -Centred Analytics, The University of Texas MD Anderson Cancer Center, Houston, Texas, USAObjective Over 160 000 participants per year complete the 12-item Oxford Hip and Knee Scores (OHS/OKS) as part of the NHS England Patient-Reported Outcome Measures (PROMs) programme. We used a modern computational approach, known as computerised adaptive testing (CAT), to simulate individually tailored OHS and OKS assessment, with the goal of reducing the number of questions a patient must complete without compromising measurement accuracy.Methods We fit the 2018/2019 PROMs data to an item response theory (IRT) model. We assessed IRT model assumptions alongside reliability. We used parameters from the IRT model with data from 2017/2018 to simulate CAT assessments. Two simulations were run until a prespecified SE of measurement was met (SE=0.32 and SE=0.45). We compared the number of questions required to meet each cut-off and assessed the correlation between the full-length and CAT administration.Results We conducted IRT analysis using 40 432 OHS and 44 714 OKS observations. The OHS and OKS were both unidimensional (root mean square error of approximation 0.08 and 0.07, respectively) and marginal reliability 0.91 and 0.90. The CAT, with a precision limit of SE=0.32 and SE=0.45, required a median of four items (IQR 1) and two items (IQR 1), respectively, for the OHS, and median of four items (IQR 2) and two items (IQR 0) for the OKS. This represents a potential 82% reduction in PROM length. In the context of 160 000 yearly assessments, these methodologies could result in the omission of some 1 280 000 redundant questions per year, which equates to 40 000 hours of patient time.Conclusion The application of IRT to the OHS and OKS produces an efficient and substantially reduced CAT. We have demonstrated a path to reduce the burden and potentially increase the compliance for these ubiquitous outcome measures without compromising measurement accuracy.https://bmjopen.bmj.com/content/12/7/e059415.full |
| spellingShingle | Jose Maria Valderas Andrew D Toms Jonathan Peter Evans Christopher Gibbons Use of computerised adaptive testing to reduce the number of items in patient-reported hip and knee outcome scores: an analysis of the NHS England National Patient-Reported Outcome Measures programme BMJ Open |
| title | Use of computerised adaptive testing to reduce the number of items in patient-reported hip and knee outcome scores: an analysis of the NHS England National Patient-Reported Outcome Measures programme |
| title_full | Use of computerised adaptive testing to reduce the number of items in patient-reported hip and knee outcome scores: an analysis of the NHS England National Patient-Reported Outcome Measures programme |
| title_fullStr | Use of computerised adaptive testing to reduce the number of items in patient-reported hip and knee outcome scores: an analysis of the NHS England National Patient-Reported Outcome Measures programme |
| title_full_unstemmed | Use of computerised adaptive testing to reduce the number of items in patient-reported hip and knee outcome scores: an analysis of the NHS England National Patient-Reported Outcome Measures programme |
| title_short | Use of computerised adaptive testing to reduce the number of items in patient-reported hip and knee outcome scores: an analysis of the NHS England National Patient-Reported Outcome Measures programme |
| title_sort | use of computerised adaptive testing to reduce the number of items in patient reported hip and knee outcome scores an analysis of the nhs england national patient reported outcome measures programme |
| url | https://bmjopen.bmj.com/content/12/7/e059415.full |
| work_keys_str_mv | AT josemariavalderas useofcomputerisedadaptivetestingtoreducethenumberofitemsinpatientreportedhipandkneeoutcomescoresananalysisofthenhsenglandnationalpatientreportedoutcomemeasuresprogramme AT andrewdtoms useofcomputerisedadaptivetestingtoreducethenumberofitemsinpatientreportedhipandkneeoutcomescoresananalysisofthenhsenglandnationalpatientreportedoutcomemeasuresprogramme AT jonathanpeterevans useofcomputerisedadaptivetestingtoreducethenumberofitemsinpatientreportedhipandkneeoutcomescoresananalysisofthenhsenglandnationalpatientreportedoutcomemeasuresprogramme AT christophergibbons useofcomputerisedadaptivetestingtoreducethenumberofitemsinpatientreportedhipandkneeoutcomescoresananalysisofthenhsenglandnationalpatientreportedoutcomemeasuresprogramme |