Exploring artificial intelligence-powered virtual assistants to understand their potential to support older adults’ search needs
Objective: We investigated the accuracy and amount of information provided by artificial intelligence (AI)-powered virtual assistants in response to queries relevant to aging adults in the domains of Medicare, long-term care insurance, and resource access. Background: Older adults are faced with com...
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025-06-01
|
| Series: | Human Factors in Healthcare |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S277250142500003X |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832586178723840000 |
|---|---|
| author | Emily M. Langston Varitnan Hattakitjamroen Mario Hernandez Hye Soo Lee Hannah Ç. Mason Willencia Louis-Charles Neil Charness Sara J. Czaja Wendy A. Rogers Joseph Sharit Walter R. Boot |
| author_facet | Emily M. Langston Varitnan Hattakitjamroen Mario Hernandez Hye Soo Lee Hannah Ç. Mason Willencia Louis-Charles Neil Charness Sara J. Czaja Wendy A. Rogers Joseph Sharit Walter R. Boot |
| author_sort | Emily M. Langston |
| collection | DOAJ |
| description | Objective: We investigated the accuracy and amount of information provided by artificial intelligence (AI)-powered virtual assistants in response to queries relevant to aging adults in the domains of Medicare, long-term care insurance, and resource access. Background: Older adults are faced with complex decisions and must gather and integrate information from diverse sources to help support these decisions (e.g., across various websites and online resources). Information-seeking, integration, and decision-making are cognitively demanding and can be impacted by age-related cognitive changes. Virtual assistants powered by AI have the potential to provide older adults with easy access to information and answers to their queries. However, it is unclear how accurate this information and these answers might be. Method: Alexa, Google Assistant, Bard, and ChatGPT-4 were queried. Coders assessed the accuracy of these responses, and the amount of supplemental information provided as a measure of response complexity. Results: Overall, Large Language Model (LLM)-based virtual assistants (Bard, ChatGPT-4) responded more accurately than non-LLM assistants (e.g., 6 % inaccurate responses for Bard vs. 60 % for Alexa) and provided substantially more supplemental information (79 % of responses with high supplemental information for Bard and 37 % for Chat-GPT, vs. 20 % or less for others). We note, however, that responses can vary over time. Conclusion: Based on their ability to provide largely accurate responses, LLMs may be helpful tools for older adults seeking information related to health, insurance, and available resources. However, the potential for error, high response complexity, and response variability should be considered. Application: LLM-based virtual assistants may be a helpful tool for older adults seeking information to support health and financial decisions. |
| format | Article |
| id | doaj-art-1cc4f59e2ef141ae98fe9e0e89ad0480 |
| institution | Kabale University |
| issn | 2772-5014 |
| language | English |
| publishDate | 2025-06-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Human Factors in Healthcare |
| spelling | doaj-art-1cc4f59e2ef141ae98fe9e0e89ad04802025-01-26T05:05:21ZengElsevierHuman Factors in Healthcare2772-50142025-06-017100092Exploring artificial intelligence-powered virtual assistants to understand their potential to support older adults’ search needsEmily M. Langston0Varitnan Hattakitjamroen1Mario Hernandez2Hye Soo Lee3Hannah Ç. Mason4Willencia Louis-Charles5Neil Charness6Sara J. Czaja7Wendy A. Rogers8Joseph Sharit9Walter R. Boot10Florida State University, Tallahassee, FL, United StatesUniversity of Illinois Urbana-Champaign, Champaign, IL, United StatesWeill Cornell Medicine, New York, NY, United StatesUniversity of Illinois Urbana-Champaign, Champaign, IL, United StatesWeill Cornell Medicine, New York, NY, United StatesUniversity of Illinois Urbana-Champaign, Champaign, IL, United StatesWeill Cornell Medicine, New York, NY, United StatesWeill Cornell Medicine, New York, NY, United StatesUniversity of Illinois Urbana-Champaign, Champaign, IL, United StatesUniversity of Miami, Miami, FL, United StatesWeill Cornell Medicine, New York, NY, United States; Corresponding author.Objective: We investigated the accuracy and amount of information provided by artificial intelligence (AI)-powered virtual assistants in response to queries relevant to aging adults in the domains of Medicare, long-term care insurance, and resource access. Background: Older adults are faced with complex decisions and must gather and integrate information from diverse sources to help support these decisions (e.g., across various websites and online resources). Information-seeking, integration, and decision-making are cognitively demanding and can be impacted by age-related cognitive changes. Virtual assistants powered by AI have the potential to provide older adults with easy access to information and answers to their queries. However, it is unclear how accurate this information and these answers might be. Method: Alexa, Google Assistant, Bard, and ChatGPT-4 were queried. Coders assessed the accuracy of these responses, and the amount of supplemental information provided as a measure of response complexity. Results: Overall, Large Language Model (LLM)-based virtual assistants (Bard, ChatGPT-4) responded more accurately than non-LLM assistants (e.g., 6 % inaccurate responses for Bard vs. 60 % for Alexa) and provided substantially more supplemental information (79 % of responses with high supplemental information for Bard and 37 % for Chat-GPT, vs. 20 % or less for others). We note, however, that responses can vary over time. Conclusion: Based on their ability to provide largely accurate responses, LLMs may be helpful tools for older adults seeking information related to health, insurance, and available resources. However, the potential for error, high response complexity, and response variability should be considered. Application: LLM-based virtual assistants may be a helpful tool for older adults seeking information to support health and financial decisions.http://www.sciencedirect.com/science/article/pii/S277250142500003XArtificial intelligenceDecision-makingAgingHealthInformation search |
| spellingShingle | Emily M. Langston Varitnan Hattakitjamroen Mario Hernandez Hye Soo Lee Hannah Ç. Mason Willencia Louis-Charles Neil Charness Sara J. Czaja Wendy A. Rogers Joseph Sharit Walter R. Boot Exploring artificial intelligence-powered virtual assistants to understand their potential to support older adults’ search needs Human Factors in Healthcare Artificial intelligence Decision-making Aging Health Information search |
| title | Exploring artificial intelligence-powered virtual assistants to understand their potential to support older adults’ search needs |
| title_full | Exploring artificial intelligence-powered virtual assistants to understand their potential to support older adults’ search needs |
| title_fullStr | Exploring artificial intelligence-powered virtual assistants to understand their potential to support older adults’ search needs |
| title_full_unstemmed | Exploring artificial intelligence-powered virtual assistants to understand their potential to support older adults’ search needs |
| title_short | Exploring artificial intelligence-powered virtual assistants to understand their potential to support older adults’ search needs |
| title_sort | exploring artificial intelligence powered virtual assistants to understand their potential to support older adults search needs |
| topic | Artificial intelligence Decision-making Aging Health Information search |
| url | http://www.sciencedirect.com/science/article/pii/S277250142500003X |
| work_keys_str_mv | AT emilymlangston exploringartificialintelligencepoweredvirtualassistantstounderstandtheirpotentialtosupportolderadultssearchneeds AT varitnanhattakitjamroen exploringartificialintelligencepoweredvirtualassistantstounderstandtheirpotentialtosupportolderadultssearchneeds AT mariohernandez exploringartificialintelligencepoweredvirtualassistantstounderstandtheirpotentialtosupportolderadultssearchneeds AT hyesoolee exploringartificialintelligencepoweredvirtualassistantstounderstandtheirpotentialtosupportolderadultssearchneeds AT hannahcmason exploringartificialintelligencepoweredvirtualassistantstounderstandtheirpotentialtosupportolderadultssearchneeds AT willencialouischarles exploringartificialintelligencepoweredvirtualassistantstounderstandtheirpotentialtosupportolderadultssearchneeds AT neilcharness exploringartificialintelligencepoweredvirtualassistantstounderstandtheirpotentialtosupportolderadultssearchneeds AT sarajczaja exploringartificialintelligencepoweredvirtualassistantstounderstandtheirpotentialtosupportolderadultssearchneeds AT wendyarogers exploringartificialintelligencepoweredvirtualassistantstounderstandtheirpotentialtosupportolderadultssearchneeds AT josephsharit exploringartificialintelligencepoweredvirtualassistantstounderstandtheirpotentialtosupportolderadultssearchneeds AT walterrboot exploringartificialintelligencepoweredvirtualassistantstounderstandtheirpotentialtosupportolderadultssearchneeds |