Evaluating the Effectiveness of Large Language Models in Converting Clinical Data to FHIR Format
The conversion of unstructured clinical data into structured formats, such as Fast Healthcare Interoperability Resources (FHIR), is a critical challenge in healthcare informatics. This study explores the potential of large language models (LLMs) to automate this conversion process, aiming to enhance...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-03-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/15/6/3379 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850205829221842944 |
|---|---|
| author | Julien Delaunay Daniel Girbes Jordi Cusido |
| author_facet | Julien Delaunay Daniel Girbes Jordi Cusido |
| author_sort | Julien Delaunay |
| collection | DOAJ |
| description | The conversion of unstructured clinical data into structured formats, such as Fast Healthcare Interoperability Resources (FHIR), is a critical challenge in healthcare informatics. This study explores the potential of large language models (LLMs) to automate this conversion process, aiming to enhance data interoperability and improve healthcare outcomes. The effectiveness of various LLMs in converting clinical reports into FHIR bundles was evaluated using different prompting techniques, including iterative correction and example-based prompting. The findings demonstrate the critical role of prompt engineering, with the two-step approach shown to significantly improve accuracy and completeness. While few-shot learning enhanced performance, it also introduced a risk of overreliance on examples. The performance of the LLMs is assessed based on the precision, hallucination rate, and resource mapping accuracy across mammography and dermatological reports from two clinics, providing insights into effective strategies for reliable FHIR data conversion and highlighting the importance of tailored prompting strategies. |
| format | Article |
| id | doaj-art-9c3473a95b194255b60d52652b8ab151 |
| institution | OA Journals |
| issn | 2076-3417 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj-art-9c3473a95b194255b60d52652b8ab1512025-08-20T02:11:00ZengMDPI AGApplied Sciences2076-34172025-03-01156337910.3390/app15063379Evaluating the Effectiveness of Large Language Models in Converting Clinical Data to FHIR FormatJulien Delaunay0Daniel Girbes1Jordi Cusido2Top Health Tech, 08035 Barcelona, SpainTop Health Tech, 08035 Barcelona, SpainTop Health Tech, 08035 Barcelona, SpainThe conversion of unstructured clinical data into structured formats, such as Fast Healthcare Interoperability Resources (FHIR), is a critical challenge in healthcare informatics. This study explores the potential of large language models (LLMs) to automate this conversion process, aiming to enhance data interoperability and improve healthcare outcomes. The effectiveness of various LLMs in converting clinical reports into FHIR bundles was evaluated using different prompting techniques, including iterative correction and example-based prompting. The findings demonstrate the critical role of prompt engineering, with the two-step approach shown to significantly improve accuracy and completeness. While few-shot learning enhanced performance, it also introduced a risk of overreliance on examples. The performance of the LLMs is assessed based on the precision, hallucination rate, and resource mapping accuracy across mammography and dermatological reports from two clinics, providing insights into effective strategies for reliable FHIR data conversion and highlighting the importance of tailored prompting strategies.https://www.mdpi.com/2076-3417/15/6/3379large language modelsFast Healthcare Interoperability Resourcesunstructured clinical datahealthcare informaticsnatural language processingprompt engineering |
| spellingShingle | Julien Delaunay Daniel Girbes Jordi Cusido Evaluating the Effectiveness of Large Language Models in Converting Clinical Data to FHIR Format Applied Sciences large language models Fast Healthcare Interoperability Resources unstructured clinical data healthcare informatics natural language processing prompt engineering |
| title | Evaluating the Effectiveness of Large Language Models in Converting Clinical Data to FHIR Format |
| title_full | Evaluating the Effectiveness of Large Language Models in Converting Clinical Data to FHIR Format |
| title_fullStr | Evaluating the Effectiveness of Large Language Models in Converting Clinical Data to FHIR Format |
| title_full_unstemmed | Evaluating the Effectiveness of Large Language Models in Converting Clinical Data to FHIR Format |
| title_short | Evaluating the Effectiveness of Large Language Models in Converting Clinical Data to FHIR Format |
| title_sort | evaluating the effectiveness of large language models in converting clinical data to fhir format |
| topic | large language models Fast Healthcare Interoperability Resources unstructured clinical data healthcare informatics natural language processing prompt engineering |
| url | https://www.mdpi.com/2076-3417/15/6/3379 |
| work_keys_str_mv | AT juliendelaunay evaluatingtheeffectivenessoflargelanguagemodelsinconvertingclinicaldatatofhirformat AT danielgirbes evaluatingtheeffectivenessoflargelanguagemodelsinconvertingclinicaldatatofhirformat AT jordicusido evaluatingtheeffectivenessoflargelanguagemodelsinconvertingclinicaldatatofhirformat |