Is Self-Mark dependable in Very Short Answer Question formats among pre-clinical medical students?
Introduction: Very Short Answer Questions (VSAQs) minimise cueing and simulate actual clinical practice more accurately than Single Best Answer Questions, as multiple-choice options might not be realistic. Phramongkutklao College of Medicine has developed a Self-Marked VSAQ (SM-VSAQ) for formative a...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
National University of Singapore
2025-04-01
|
| Series: | The Asia Pacific Scholar |
| Subjects: | |
| Online Access: | https://medicine.nus.edu.sg/taps/is-self-mark-dependable-in-very-short-answer-question-formats-among-pre-clinical-medical-students/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | Introduction: Very Short Answer Questions (VSAQs) minimise cueing and simulate actual clinical practice more accurately than Single Best Answer Questions, as multiple-choice options might not be realistic. Phramongkutklao College of Medicine has developed a Self-Marked VSAQ (SM-VSAQ) for formative assessments. This study determines the validity and reliability of the SM-VSAQs.
Methods: Ninety-four third-year pre-clinical students took two occasions of 10-question SM-VSAQ exams regarding cardiovascular drugs. Each question consisted of two steps: (1) clinical vignettes with questions and (2) expected answers with scores, self-marking, and feedback comprehension. Scores ranged from 0.00 to 1.00 in 0.25 increments, though not every increment was applied to all questions. A distribution of the rating agreement between students' and teacher’s ratings was presented to determine criterion-related validity and inter-rater reliability.
Results: Criterion-related validity revealed 90.64% and 93.19% of the ratings demonstrated exact agreement between students' and teachers' ratings, with an inter-rater reliability of 0.972 and 0.977 for the first and second occasions, respectively (p=0.001). The exact agreement was relatively lower on the first occasion for questions with more diverse expected answers (85.11%, r=0.867, p=0.001) and drugs requiring their specific full names for a perfect mark (74.47%, r=0.849, p=0.001). While questions with specific guides do not require complex answers, they received a higher exact agreement.
Conclusion: The SM-VSAQ format effectively combines guided answers with the VSAQ model. The agreement with teacher-rated is excellent. Marking discrepancies rooted in misconceptions underscores the importance of teacher feedback in improving self-grading in formative assessments. Regular self-assessment practice is recommended to enhance grading accuracy. |
|---|---|
| ISSN: | 2424-9335 2424-9270 |