11 Key Strategies in Using AI Chat GPT to Develop HOTS-Based Entrepreneurship Questionnaires
Developing entrepreneurship assessment questions that measure higher-order thinking skills (HOTS) is a challenge that requires time and in-depth analysis. Artificial intelligence such as Chat GPT is here to simplify the process. However, its use in creating HOTS-based questions in entrepreneurship...
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Qubahan
2025-01-01
|
| Series: | Qubahan Academic Journal |
| Online Access: | https://journal.qubahan.com/index.php/qaj/article/view/1134 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832544582621986816 |
|---|---|
| author | Almaida Vebibina Rusijono Rusijono Andi Mariono Ahsan Muafa Dydik Kurniawan Ragil Sugeng Dewantoro Dian Noer Asyari Sussi Widiastuti Jarot Suseno |
| author_facet | Almaida Vebibina Rusijono Rusijono Andi Mariono Ahsan Muafa Dydik Kurniawan Ragil Sugeng Dewantoro Dian Noer Asyari Sussi Widiastuti Jarot Suseno |
| author_sort | Almaida Vebibina |
| collection | DOAJ |
| description |
Developing entrepreneurship assessment questions that measure higher-order thinking skills (HOTS) is a challenge that requires time and in-depth analysis. Artificial intelligence such as Chat GPT is here to simplify the process. However, its use in creating HOTS-based questions in entrepreneurship still faces obstacles. This research is proposed based on the background of various obstacles in the use of Chat GPT to develop an effective questionnaire that meets the learning objectives. To bridge the gap, this study explores the potential of Chat GPT in crafting assessment questions that focus on higher-order thinking by utilizing existing strategies to enhance critical, analytical, creative and problem-solving thinking in entrepreneurship education. A total of 20 entrepreneurship experts at Surabaya State University were involved in this research. We developed LLM (Large Language Model) using the approach to create HOTS-based multiple-choice questions (MCQs) from an advanced course context. Subsequently, we evaluated 470 MCQs generated by Chat GPT-4 and 337 MCQs generated by Chat GPT-3.5 as well as 595 MCQs corresponding to higher-order cognitive levels. An assessment instrument was used to evaluate the quality of questions generated through 23 items adopted from previous studies. To test the difference in performance, the Mann-Whitney Test was analysed with a p value < 0.05. The results showed that ChatGPT-4 was able to generate questions with clear language usage, providing a single correct answer, as well as a high-quality selection of distractors. In addition, the questions developed by ChatGPT-4 are in accordance with HOTS aspects. This finding can be utilized by teachers as the latest generative model to increase the bank of diverse multiple-choice questions without replacing human expertise.
|
| format | Article |
| id | doaj-art-e468104258e148cbb1d1321c985bcc0e |
| institution | Kabale University |
| issn | 2709-8206 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Qubahan |
| record_format | Article |
| series | Qubahan Academic Journal |
| spelling | doaj-art-e468104258e148cbb1d1321c985bcc0e2025-02-03T10:11:15ZengQubahanQubahan Academic Journal2709-82062025-01-015110.48161/qaj.v5n1a1134113411 Key Strategies in Using AI Chat GPT to Develop HOTS-Based Entrepreneurship Questionnaires Almaida Vebibina0Rusijono Rusijono 1Andi Mariono 2Ahsan Muafa 3Dydik Kurniawan 4Ragil Sugeng Dewantoro5Dian Noer Asyari 6Sussi Widiastuti 7Jarot Suseno 8Department of Educational technology, Faculty of Education, Universitas Negeri Surabaya, Surabaya 60213, Indonesia; Department of Cosmetology Education, Faculty of Engineering, Universitas Negeri Medan, Medan 20222, Indonesia.Department of Educational technology, Faculty of Education, Universitas Negeri Surabaya, Surabaya 60213, Indonesia; Department of Educational technology, Faculty of Education, Universitas Negeri Surabaya, Surabaya 60213, Indonesia; Department of Educational technology, Faculty of Education, Universitas Negeri Surabaya, Surabaya 60213, Indonesia; Department of Educational technology, Faculty of Education, Universitas Negeri Surabaya, Surabaya 60213, Indonesia; Department of Educational technology, Faculty of Education, Universitas Negeri Surabaya, Surabaya 60213, Indonesia; Department of Educational technology, Faculty of Education, Universitas Negeri Surabaya, Surabaya 60213, Indonesia; Department of Educational technology, Faculty of Education, Universitas Negeri Surabaya, Surabaya 60213, Indonesia; Department of Educational technology, Faculty of Education, Universitas Negeri Surabaya, Surabaya 60213, Indonesia; Developing entrepreneurship assessment questions that measure higher-order thinking skills (HOTS) is a challenge that requires time and in-depth analysis. Artificial intelligence such as Chat GPT is here to simplify the process. However, its use in creating HOTS-based questions in entrepreneurship still faces obstacles. This research is proposed based on the background of various obstacles in the use of Chat GPT to develop an effective questionnaire that meets the learning objectives. To bridge the gap, this study explores the potential of Chat GPT in crafting assessment questions that focus on higher-order thinking by utilizing existing strategies to enhance critical, analytical, creative and problem-solving thinking in entrepreneurship education. A total of 20 entrepreneurship experts at Surabaya State University were involved in this research. We developed LLM (Large Language Model) using the approach to create HOTS-based multiple-choice questions (MCQs) from an advanced course context. Subsequently, we evaluated 470 MCQs generated by Chat GPT-4 and 337 MCQs generated by Chat GPT-3.5 as well as 595 MCQs corresponding to higher-order cognitive levels. An assessment instrument was used to evaluate the quality of questions generated through 23 items adopted from previous studies. To test the difference in performance, the Mann-Whitney Test was analysed with a p value < 0.05. The results showed that ChatGPT-4 was able to generate questions with clear language usage, providing a single correct answer, as well as a high-quality selection of distractors. In addition, the questions developed by ChatGPT-4 are in accordance with HOTS aspects. This finding can be utilized by teachers as the latest generative model to increase the bank of diverse multiple-choice questions without replacing human expertise. https://journal.qubahan.com/index.php/qaj/article/view/1134 |
| spellingShingle | Almaida Vebibina Rusijono Rusijono Andi Mariono Ahsan Muafa Dydik Kurniawan Ragil Sugeng Dewantoro Dian Noer Asyari Sussi Widiastuti Jarot Suseno 11 Key Strategies in Using AI Chat GPT to Develop HOTS-Based Entrepreneurship Questionnaires Qubahan Academic Journal |
| title | 11 Key Strategies in Using AI Chat GPT to Develop HOTS-Based Entrepreneurship Questionnaires |
| title_full | 11 Key Strategies in Using AI Chat GPT to Develop HOTS-Based Entrepreneurship Questionnaires |
| title_fullStr | 11 Key Strategies in Using AI Chat GPT to Develop HOTS-Based Entrepreneurship Questionnaires |
| title_full_unstemmed | 11 Key Strategies in Using AI Chat GPT to Develop HOTS-Based Entrepreneurship Questionnaires |
| title_short | 11 Key Strategies in Using AI Chat GPT to Develop HOTS-Based Entrepreneurship Questionnaires |
| title_sort | 11 key strategies in using ai chat gpt to develop hots based entrepreneurship questionnaires |
| url | https://journal.qubahan.com/index.php/qaj/article/view/1134 |
| work_keys_str_mv | AT almaidavebibina 11keystrategiesinusingaichatgpttodevelophotsbasedentrepreneurshipquestionnaires AT rusijonorusijono 11keystrategiesinusingaichatgpttodevelophotsbasedentrepreneurshipquestionnaires AT andimariono 11keystrategiesinusingaichatgpttodevelophotsbasedentrepreneurshipquestionnaires AT ahsanmuafa 11keystrategiesinusingaichatgpttodevelophotsbasedentrepreneurshipquestionnaires AT dydikkurniawan 11keystrategiesinusingaichatgpttodevelophotsbasedentrepreneurshipquestionnaires AT ragilsugengdewantoro 11keystrategiesinusingaichatgpttodevelophotsbasedentrepreneurshipquestionnaires AT diannoerasyari 11keystrategiesinusingaichatgpttodevelophotsbasedentrepreneurshipquestionnaires AT sussiwidiastuti 11keystrategiesinusingaichatgpttodevelophotsbasedentrepreneurshipquestionnaires AT jarotsuseno 11keystrategiesinusingaichatgpttodevelophotsbasedentrepreneurshipquestionnaires |