Multimodal Assessment of Mental Workload During Automated Vehicle Remote Assistance: Modeling of Eye-Tracking-Related, Skin Conductance, and Cardiovascular Indicators
Remote assistance for highly automated vehicles (HAVs), i.e., third-party assistance from support staff outside the vehicle in times of the need for assistance, presents a solution to extend the capabilities of HAVs by integrating a third party for decision making in uncertain situations. Similar to...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-01-01
|
| Series: | Information |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2078-2489/16/1/64 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832588370003361792 |
|---|---|
| author | Fabian Walocha Andreas Schrank Hoai Phuong Nguyen Klas Ihme |
| author_facet | Fabian Walocha Andreas Schrank Hoai Phuong Nguyen Klas Ihme |
| author_sort | Fabian Walocha |
| collection | DOAJ |
| description | Remote assistance for highly automated vehicles (HAVs), i.e., third-party assistance from support staff outside the vehicle in times of the need for assistance, presents a solution to extend the capabilities of HAVs by integrating a third party for decision making in uncertain situations. Similar to other control center positions, we expect the remote assistance tasks to exert high mental demands on the human operators. Therefore, we assessed impact of elevated mental workload during HAV remote assistance in a controlled environment in a user study (N = 37) with the goal of identifying cues to differentiate workload levels based on eye-tracking-related, skin conductance, and cardiovascular indicators. The results provide evidence that (A) elevated workload induced via a secondary task depreciates performance, and (B) we can identify workload levels person-independently as differences in tonic skin conductance (F(2,72) = 24.538, <i>p</i> < 0.001, partial η² = 0.405) and pupil dilation (F(2,72) = 13.872, <i>p</i> < 0.001, partial η² = 0.278), resulting in a classification accuracy of 58% in a three-class classification task. The results provide evidence that we are able to differentiate operator workload during remote assistance in a time-resolved way with the ultimate goal to provide adaptations to counteract task deficiencies. |
| format | Article |
| id | doaj-art-bdc61ffa3198493caa6f869483670154 |
| institution | Kabale University |
| issn | 2078-2489 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Information |
| spelling | doaj-art-bdc61ffa3198493caa6f8694836701542025-01-24T13:35:19ZengMDPI AGInformation2078-24892025-01-011616410.3390/info16010064Multimodal Assessment of Mental Workload During Automated Vehicle Remote Assistance: Modeling of Eye-Tracking-Related, Skin Conductance, and Cardiovascular IndicatorsFabian Walocha0Andreas Schrank1Hoai Phuong Nguyen2Klas Ihme3German Aerospace Center, Institute of Transportation Systems, 38108 Braunschweig, GermanyGerman Aerospace Center, Institute of Transportation Systems, 38108 Braunschweig, GermanyGerman Aerospace Center, Institute of Transportation Systems, 38108 Braunschweig, GermanyGerman Aerospace Center, Institute of Transportation Systems, 38108 Braunschweig, GermanyRemote assistance for highly automated vehicles (HAVs), i.e., third-party assistance from support staff outside the vehicle in times of the need for assistance, presents a solution to extend the capabilities of HAVs by integrating a third party for decision making in uncertain situations. Similar to other control center positions, we expect the remote assistance tasks to exert high mental demands on the human operators. Therefore, we assessed impact of elevated mental workload during HAV remote assistance in a controlled environment in a user study (N = 37) with the goal of identifying cues to differentiate workload levels based on eye-tracking-related, skin conductance, and cardiovascular indicators. The results provide evidence that (A) elevated workload induced via a secondary task depreciates performance, and (B) we can identify workload levels person-independently as differences in tonic skin conductance (F(2,72) = 24.538, <i>p</i> < 0.001, partial η² = 0.405) and pupil dilation (F(2,72) = 13.872, <i>p</i> < 0.001, partial η² = 0.278), resulting in a classification accuracy of 58% in a three-class classification task. The results provide evidence that we are able to differentiate operator workload during remote assistance in a time-resolved way with the ultimate goal to provide adaptations to counteract task deficiencies.https://www.mdpi.com/2078-2489/16/1/64remote operationautonomous vehiclesremote assistanceuser state monitoringmental workloadphysiology |
| spellingShingle | Fabian Walocha Andreas Schrank Hoai Phuong Nguyen Klas Ihme Multimodal Assessment of Mental Workload During Automated Vehicle Remote Assistance: Modeling of Eye-Tracking-Related, Skin Conductance, and Cardiovascular Indicators Information remote operation autonomous vehicles remote assistance user state monitoring mental workload physiology |
| title | Multimodal Assessment of Mental Workload During Automated Vehicle Remote Assistance: Modeling of Eye-Tracking-Related, Skin Conductance, and Cardiovascular Indicators |
| title_full | Multimodal Assessment of Mental Workload During Automated Vehicle Remote Assistance: Modeling of Eye-Tracking-Related, Skin Conductance, and Cardiovascular Indicators |
| title_fullStr | Multimodal Assessment of Mental Workload During Automated Vehicle Remote Assistance: Modeling of Eye-Tracking-Related, Skin Conductance, and Cardiovascular Indicators |
| title_full_unstemmed | Multimodal Assessment of Mental Workload During Automated Vehicle Remote Assistance: Modeling of Eye-Tracking-Related, Skin Conductance, and Cardiovascular Indicators |
| title_short | Multimodal Assessment of Mental Workload During Automated Vehicle Remote Assistance: Modeling of Eye-Tracking-Related, Skin Conductance, and Cardiovascular Indicators |
| title_sort | multimodal assessment of mental workload during automated vehicle remote assistance modeling of eye tracking related skin conductance and cardiovascular indicators |
| topic | remote operation autonomous vehicles remote assistance user state monitoring mental workload physiology |
| url | https://www.mdpi.com/2078-2489/16/1/64 |
| work_keys_str_mv | AT fabianwalocha multimodalassessmentofmentalworkloadduringautomatedvehicleremoteassistancemodelingofeyetrackingrelatedskinconductanceandcardiovascularindicators AT andreasschrank multimodalassessmentofmentalworkloadduringautomatedvehicleremoteassistancemodelingofeyetrackingrelatedskinconductanceandcardiovascularindicators AT hoaiphuongnguyen multimodalassessmentofmentalworkloadduringautomatedvehicleremoteassistancemodelingofeyetrackingrelatedskinconductanceandcardiovascularindicators AT klasihme multimodalassessmentofmentalworkloadduringautomatedvehicleremoteassistancemodelingofeyetrackingrelatedskinconductanceandcardiovascularindicators |