ChatGPT as a data analyst: an exploratory study on AI-supported quantitative data analysis in empirical research
Social scientists are faced with the challenge of designing complex studies and analyzing collected data via various programs such as R, Stata, SPSS, or Python. This often requires the use of analytical procedures and specific software packages that are beyond an individual’s established skillsets a...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2025-01-01
|
| Series: | Frontiers in Education |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/feduc.2024.1417900/full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832591277721387008 |
|---|---|
| author | Dimitri Prandner Daniela Wetzelhütter Sönke Hese |
| author_facet | Dimitri Prandner Daniela Wetzelhütter Sönke Hese |
| author_sort | Dimitri Prandner |
| collection | DOAJ |
| description | Social scientists are faced with the challenge of designing complex studies and analyzing collected data via various programs such as R, Stata, SPSS, or Python. This often requires the use of analytical procedures and specific software packages that are beyond an individual’s established skillsets and technical knowledge. To address these challenges, generative artificial intelligence, such as ChatGPT, can now be employed as ‘assistants’—with both associated risks and benefits. Accordingly, this paper explores the potential and pitfalls of using a tool like ChatGPT as an assistant in quantitative data analysis. We investigate the practical use of ChatGPT-3.5 by replicating analyses and findings in everyday scientific research. Unlike previous studies, which have primarily focused optimizing the use of chatbots for code generation, our approach examines an amateur level use of AI tools to support and reference regular research activities, with an emphasis on minimal technical expertise. While we overall conducted three experiments, with the goal to replicate academic papers, the article’s focus is on the methodologically most complex one, by De Wet et al. from 2020. In this case AI is used for the step-by-step replication of the two-dimensional model of value types proposed by Schwartz (2012). The results of this experiment highlight the challenges of using ChatGPT 3.5 for specific, detailed tasks in academic research, as a tendency for responses to repeat in loops when solutions were not readily available emerged at several stages. Thus, we concluded that there are severe limitations in the AI’s ability to provide accurate and comprehensive solutions for complex tasks and emphasize the need for caution and verification when using AI powered tools for complex research procedures. |
| format | Article |
| id | doaj-art-fd49628492ac42d187bb178ee79ba331 |
| institution | Kabale University |
| issn | 2504-284X |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Education |
| spelling | doaj-art-fd49628492ac42d187bb178ee79ba3312025-01-22T14:55:33ZengFrontiers Media S.A.Frontiers in Education2504-284X2025-01-01910.3389/feduc.2024.14179001417900ChatGPT as a data analyst: an exploratory study on AI-supported quantitative data analysis in empirical researchDimitri Prandner0Daniela Wetzelhütter1Sönke Hese2Johannes Kepler University, Institute of Sociology, Department of Empirical Social Research, Linz, AustriaDepartment for Social Work, School of Medical Engineering and Applied Social Sciences, University of Applied Sciences Upper Austria, Linz, AustriaInstitute for Social Sciences / Sociology, Christian-Albrechts-Universität zu Kiel, Kiel, GermanySocial scientists are faced with the challenge of designing complex studies and analyzing collected data via various programs such as R, Stata, SPSS, or Python. This often requires the use of analytical procedures and specific software packages that are beyond an individual’s established skillsets and technical knowledge. To address these challenges, generative artificial intelligence, such as ChatGPT, can now be employed as ‘assistants’—with both associated risks and benefits. Accordingly, this paper explores the potential and pitfalls of using a tool like ChatGPT as an assistant in quantitative data analysis. We investigate the practical use of ChatGPT-3.5 by replicating analyses and findings in everyday scientific research. Unlike previous studies, which have primarily focused optimizing the use of chatbots for code generation, our approach examines an amateur level use of AI tools to support and reference regular research activities, with an emphasis on minimal technical expertise. While we overall conducted three experiments, with the goal to replicate academic papers, the article’s focus is on the methodologically most complex one, by De Wet et al. from 2020. In this case AI is used for the step-by-step replication of the two-dimensional model of value types proposed by Schwartz (2012). The results of this experiment highlight the challenges of using ChatGPT 3.5 for specific, detailed tasks in academic research, as a tendency for responses to repeat in loops when solutions were not readily available emerged at several stages. Thus, we concluded that there are severe limitations in the AI’s ability to provide accurate and comprehensive solutions for complex tasks and emphasize the need for caution and verification when using AI powered tools for complex research procedures.https://www.frontiersin.org/articles/10.3389/feduc.2024.1417900/fullChatGPTdata analystAI-supportedquantitativedata analysis |
| spellingShingle | Dimitri Prandner Daniela Wetzelhütter Sönke Hese ChatGPT as a data analyst: an exploratory study on AI-supported quantitative data analysis in empirical research Frontiers in Education ChatGPT data analyst AI-supported quantitative data analysis |
| title | ChatGPT as a data analyst: an exploratory study on AI-supported quantitative data analysis in empirical research |
| title_full | ChatGPT as a data analyst: an exploratory study on AI-supported quantitative data analysis in empirical research |
| title_fullStr | ChatGPT as a data analyst: an exploratory study on AI-supported quantitative data analysis in empirical research |
| title_full_unstemmed | ChatGPT as a data analyst: an exploratory study on AI-supported quantitative data analysis in empirical research |
| title_short | ChatGPT as a data analyst: an exploratory study on AI-supported quantitative data analysis in empirical research |
| title_sort | chatgpt as a data analyst an exploratory study on ai supported quantitative data analysis in empirical research |
| topic | ChatGPT data analyst AI-supported quantitative data analysis |
| url | https://www.frontiersin.org/articles/10.3389/feduc.2024.1417900/full |
| work_keys_str_mv | AT dimitriprandner chatgptasadataanalystanexploratorystudyonaisupportedquantitativedataanalysisinempiricalresearch AT danielawetzelhutter chatgptasadataanalystanexploratorystudyonaisupportedquantitativedataanalysisinempiricalresearch AT sonkehese chatgptasadataanalystanexploratorystudyonaisupportedquantitativedataanalysisinempiricalresearch |