Differences in User Perception of Artificial Intelligence-Driven Chatbots and Traditional Tools in Qualitative Data Analysis
Qualitative data analysis (QDA) tools are essential for extracting insights from complex datasets. This study investigates researchers’ perceptions of the usability, user experience (UX), mental workload, trust, task complexity, and emotional impact of three tools: Taguette 1.4.1 (a traditional QDA...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-01-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/15/2/631 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832589299543965696 |
|---|---|
| author | Boštjan Šumak Maja Pušnik Ines Kožuh Andrej Šorgo Saša Brdnik |
| author_facet | Boštjan Šumak Maja Pušnik Ines Kožuh Andrej Šorgo Saša Brdnik |
| author_sort | Boštjan Šumak |
| collection | DOAJ |
| description | Qualitative data analysis (QDA) tools are essential for extracting insights from complex datasets. This study investigates researchers’ perceptions of the usability, user experience (UX), mental workload, trust, task complexity, and emotional impact of three tools: Taguette 1.4.1 (a traditional QDA tool), ChatGPT (GPT-4, December 2023 version), and Gemini (formerly Google Bard, December 2023 version). Participants (N = 85), Master’s students from the Faculty of Electrical Engineering and Computer Science with prior experience in UX evaluations and familiarity with AI-based chatbots, performed sentiment analysis and data annotation tasks using these tools, enabling a comparative evaluation. The results show that AI tools were associated with lower cognitive effort and more positive emotional responses compared to Taguette, which caused higher frustration and workload, especially during cognitively demanding tasks. Among the tools, ChatGPT achieved the highest usability score (SUS = 79.03) and was rated positively for emotional engagement. Trust levels varied, with Taguette preferred for task accuracy and ChatGPT rated highest in user confidence. Despite these differences, all tools performed consistently in identifying qualitative patterns. These findings suggest that AI-driven tools can enhance researchers’ experiences in QDA while emphasizing the need to align tool selection with specific tasks and user preferences. |
| format | Article |
| id | doaj-art-ed84b393c8924929873ca5e0e8f4ef7d |
| institution | Kabale University |
| issn | 2076-3417 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj-art-ed84b393c8924929873ca5e0e8f4ef7d2025-01-24T13:20:12ZengMDPI AGApplied Sciences2076-34172025-01-0115263110.3390/app15020631Differences in User Perception of Artificial Intelligence-Driven Chatbots and Traditional Tools in Qualitative Data AnalysisBoštjan Šumak0Maja Pušnik1Ines Kožuh2Andrej Šorgo3Saša Brdnik4Faculty of Electrical Engineering and Computer Science, University of Maribor, 2000 Maribor, SloveniaFaculty of Electrical Engineering and Computer Science, University of Maribor, 2000 Maribor, SloveniaFaculty of Electrical Engineering and Computer Science, University of Maribor, 2000 Maribor, SloveniaFaculty of Natural Sciences and Mathematics, University of Maribor, 2000 Maribor, SloveniaFaculty of Electrical Engineering and Computer Science, University of Maribor, 2000 Maribor, SloveniaQualitative data analysis (QDA) tools are essential for extracting insights from complex datasets. This study investigates researchers’ perceptions of the usability, user experience (UX), mental workload, trust, task complexity, and emotional impact of three tools: Taguette 1.4.1 (a traditional QDA tool), ChatGPT (GPT-4, December 2023 version), and Gemini (formerly Google Bard, December 2023 version). Participants (N = 85), Master’s students from the Faculty of Electrical Engineering and Computer Science with prior experience in UX evaluations and familiarity with AI-based chatbots, performed sentiment analysis and data annotation tasks using these tools, enabling a comparative evaluation. The results show that AI tools were associated with lower cognitive effort and more positive emotional responses compared to Taguette, which caused higher frustration and workload, especially during cognitively demanding tasks. Among the tools, ChatGPT achieved the highest usability score (SUS = 79.03) and was rated positively for emotional engagement. Trust levels varied, with Taguette preferred for task accuracy and ChatGPT rated highest in user confidence. Despite these differences, all tools performed consistently in identifying qualitative patterns. These findings suggest that AI-driven tools can enhance researchers’ experiences in QDA while emphasizing the need to align tool selection with specific tasks and user preferences.https://www.mdpi.com/2076-3417/15/2/631user experienceUXusabilityqualitative data analysisQDAchatbots |
| spellingShingle | Boštjan Šumak Maja Pušnik Ines Kožuh Andrej Šorgo Saša Brdnik Differences in User Perception of Artificial Intelligence-Driven Chatbots and Traditional Tools in Qualitative Data Analysis Applied Sciences user experience UX usability qualitative data analysis QDA chatbots |
| title | Differences in User Perception of Artificial Intelligence-Driven Chatbots and Traditional Tools in Qualitative Data Analysis |
| title_full | Differences in User Perception of Artificial Intelligence-Driven Chatbots and Traditional Tools in Qualitative Data Analysis |
| title_fullStr | Differences in User Perception of Artificial Intelligence-Driven Chatbots and Traditional Tools in Qualitative Data Analysis |
| title_full_unstemmed | Differences in User Perception of Artificial Intelligence-Driven Chatbots and Traditional Tools in Qualitative Data Analysis |
| title_short | Differences in User Perception of Artificial Intelligence-Driven Chatbots and Traditional Tools in Qualitative Data Analysis |
| title_sort | differences in user perception of artificial intelligence driven chatbots and traditional tools in qualitative data analysis |
| topic | user experience UX usability qualitative data analysis QDA chatbots |
| url | https://www.mdpi.com/2076-3417/15/2/631 |
| work_keys_str_mv | AT bostjansumak differencesinuserperceptionofartificialintelligencedrivenchatbotsandtraditionaltoolsinqualitativedataanalysis AT majapusnik differencesinuserperceptionofartificialintelligencedrivenchatbotsandtraditionaltoolsinqualitativedataanalysis AT ineskozuh differencesinuserperceptionofartificialintelligencedrivenchatbotsandtraditionaltoolsinqualitativedataanalysis AT andrejsorgo differencesinuserperceptionofartificialintelligencedrivenchatbotsandtraditionaltoolsinqualitativedataanalysis AT sasabrdnik differencesinuserperceptionofartificialintelligencedrivenchatbotsandtraditionaltoolsinqualitativedataanalysis |