A positive spin: large language models can help directors evaluate programs through their patients' own words
Abstract. Introduction:. Interpretation and utilization of qualitative feedback from participants has immense value for program evaluation. Reliance on only quantitative data runs the risk of losing the lived patient experience, forcing their outcomes to fit into our predefined objectives. Objective...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wolters Kluwer
2025-02-01
|
| Series: | PAIN Reports |
| Online Access: | http://journals.lww.com/painrpts/fulltext/10.1097/PR9.0000000000001219 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832589674318659584 |
|---|---|
| author | Leah Russell Flaherty Kendra H. Oliver |
| author_facet | Leah Russell Flaherty Kendra H. Oliver |
| author_sort | Leah Russell Flaherty |
| collection | DOAJ |
| description | Abstract. Introduction:. Interpretation and utilization of qualitative feedback from participants has immense value for program evaluation. Reliance on only quantitative data runs the risk of losing the lived patient experience, forcing their outcomes to fit into our predefined objectives.
Objectives:. Using large language models (LLMs), program directors may begin to employ rich, qualitative feedback expediently.
Methods:. This study provides an example of the feasibility of evaluating patient responses (n = 82) to Empowered Relief, a skill-based pain education class using LLMs. We utilized a dual-method analytical approach, with both LLM-assisted and supported manual thematic review.
Results:. The thematic analysis of qualitative data using ChatGPT yielded 7 major themes: (1) Use of Specific Audiofile; (2) Mindset; (3) Technique; (4) Community and Space; (5) Knowledge; (6) Tools and Approaches; and (7) Self-awareness.
Conclusion:. Findings from the LLM-derived analysis provided rich and unexpected information, valuable to the program and the field of pain psychology by employing the set of patients' own words to guide program evaluation. Program directors may benefit from evaluating treatment outcomes on a broader scale such as this rather than focusing solely on improvements in disability. These insights would only be uncovered with open-ended data, and although potentially more insights could emerge with the help of a qualitative research team, ChatGPT offered an ergonomic solution. |
| format | Article |
| id | doaj-art-ffed10a0348d4bc2a9dcc0dee22e5eaf |
| institution | Kabale University |
| issn | 2471-2531 |
| language | English |
| publishDate | 2025-02-01 |
| publisher | Wolters Kluwer |
| record_format | Article |
| series | PAIN Reports |
| spelling | doaj-art-ffed10a0348d4bc2a9dcc0dee22e5eaf2025-01-24T09:20:44ZengWolters KluwerPAIN Reports2471-25312025-02-01101e121910.1097/PR9.0000000000001219PR90000000000001219A positive spin: large language models can help directors evaluate programs through their patients' own wordsLeah Russell Flaherty0Kendra H. Oliver1a Allegheny Health Network, Psychiatry and Behavioral Health Instiute, Pittsburgh, PA, USAb Responsible Data Science, Office of the Provost, University of Pittsburgh, Pittsburgh, PA, USAAbstract. Introduction:. Interpretation and utilization of qualitative feedback from participants has immense value for program evaluation. Reliance on only quantitative data runs the risk of losing the lived patient experience, forcing their outcomes to fit into our predefined objectives. Objectives:. Using large language models (LLMs), program directors may begin to employ rich, qualitative feedback expediently. Methods:. This study provides an example of the feasibility of evaluating patient responses (n = 82) to Empowered Relief, a skill-based pain education class using LLMs. We utilized a dual-method analytical approach, with both LLM-assisted and supported manual thematic review. Results:. The thematic analysis of qualitative data using ChatGPT yielded 7 major themes: (1) Use of Specific Audiofile; (2) Mindset; (3) Technique; (4) Community and Space; (5) Knowledge; (6) Tools and Approaches; and (7) Self-awareness. Conclusion:. Findings from the LLM-derived analysis provided rich and unexpected information, valuable to the program and the field of pain psychology by employing the set of patients' own words to guide program evaluation. Program directors may benefit from evaluating treatment outcomes on a broader scale such as this rather than focusing solely on improvements in disability. These insights would only be uncovered with open-ended data, and although potentially more insights could emerge with the help of a qualitative research team, ChatGPT offered an ergonomic solution.http://journals.lww.com/painrpts/fulltext/10.1097/PR9.0000000000001219 |
| spellingShingle | Leah Russell Flaherty Kendra H. Oliver A positive spin: large language models can help directors evaluate programs through their patients' own words PAIN Reports |
| title | A positive spin: large language models can help directors evaluate programs through their patients' own words |
| title_full | A positive spin: large language models can help directors evaluate programs through their patients' own words |
| title_fullStr | A positive spin: large language models can help directors evaluate programs through their patients' own words |
| title_full_unstemmed | A positive spin: large language models can help directors evaluate programs through their patients' own words |
| title_short | A positive spin: large language models can help directors evaluate programs through their patients' own words |
| title_sort | positive spin large language models can help directors evaluate programs through their patients own words |
| url | http://journals.lww.com/painrpts/fulltext/10.1097/PR9.0000000000001219 |
| work_keys_str_mv | AT leahrussellflaherty apositivespinlargelanguagemodelscanhelpdirectorsevaluateprogramsthroughtheirpatientsownwords AT kendraholiver apositivespinlargelanguagemodelscanhelpdirectorsevaluateprogramsthroughtheirpatientsownwords AT leahrussellflaherty positivespinlargelanguagemodelscanhelpdirectorsevaluateprogramsthroughtheirpatientsownwords AT kendraholiver positivespinlargelanguagemodelscanhelpdirectorsevaluateprogramsthroughtheirpatientsownwords |