AI vs. teacher feedback on EFL argumentative writing: a quantitative study by Areen Alnemrat, Hesham Aldamen, Mohamad Almashour, Mutasim Al-Deaibes, Rami AlSharefeen

Get full text

RVBench: Role values benchmark for role-playing LLMs by Ye Wang, Tong Li, Meixuan Li, Ziyue Cheng, Ge Wang, Hanyue Kang, Yaling Deng, Hongjiang Xiao, Yuan Zhang

“…With the explosive development of Large Language Models (LLMs), the demand for role-playing agents has greatly increased to promote applications such as personalized digital companion and artificial society simulation. …”
Get full text

Evaluating Translation Quality: A Qualitative and Quantitative Assessment of Machine and LLM-Driven Arabic–English Translations by Tawffeek A. S. Mohammed

Get full text

On the Validity of Traditional Vulnerability Scoring Systems for Adversarial Attacks Against LLMs by Atmane Ayoub Mansour Bahar, Ahmad Samer Wazan

“…This study adopts a quantitative approach, calculating and comparing the coefficient of variation of vulnerability scores across 56 adversarial attacks on LLMs. …”
Get full text

Medical reasoning in LLMs: an in-depth analysis of DeepSeek R1 by Birger Moëll, Fredrik Sand Aronsson, Fredrik Sand Aronsson, Sanian Akbar

“…IntroductionThe integration of large language models (LLMs) into healthcare holds immense promise, but also raises critical challenges, particularly regarding the interpretability and reliability of their reasoning processes. …”
Get full text

Scaling hermeneutics: a guide to qualitative coding with LLMs for reflexive content analysis by Zackary Okun Dunivin

“…As large language models (LLMs) become increasingly adept at nuanced language tasks, questions arise about whether—and how—they can assist in large-scale coding without eroding the interpretive depth that distinguishes qualitative analysis from traditional machine learning and other quantitative approaches to natural language processing. …”
Get full text

Entropy-Optimized Dynamic Text Segmentation and RAG-Enhanced LLMs for Construction Engineering Knowledge Base by Haiyuan Wang, Deli Zhang, Jianmin Li, Zelong Feng, Feng Zhang

“…The advancement of large language models (LLMs) and Retrieval-Augmented Generation (RAG) technologies provides robust technical support for constructing domain-specific knowledge bases. …”
Get full text

AutoTA: A Dynamic Intent-Based Virtual Teaching Assistant for Students Using Open Source LLMs by Rajashree Dahal, Greg Murray, Robin Chataut, Mohamed Hefeida, Anurag K. Srivastava, Prashnna K. Gyawali

“…To evaluate the framework, we used course materials from the undergraduate-level CS course, Computer Incident Response, and compared the performance of several open-source LLMs, including Llama 3.1. Our results show that the framework accurately classifies intent and provides appropriate guidance, measured through quantitative and qualitative metrics. …”
Get full text

Enhancing stock timing predictions based on multimodal architecture: Leveraging large language models (LLMs) for text quality improvement. by Mingming Chen, Yifan Tang, Qi Qi, Hongyi Dai, Yi Lin, Chengxiu Ling, Tenglong Li

“…This study aims to enhance stock timing predictions by leveraging large language models (LLMs), specifically GPT-4, to filter and analyze online investor comment data. …”
Get full text

Evaluating gender bias in large language models in long-term care by Sam Rickman

“…However, the findings highlight the variation in state-of-the-art LLMs, and the need for evaluation of bias. The methods in this paper provide a practical framework for quantitative evaluation of gender bias in LLMs. …”
Get full text

Conversational LLM-Based Decision Support for Defect Classification in AFM Images by Angona Biswas, Jaydeep Rade, Nabila Masud, Md Hasibul Hasan Hasib, Aditya Balu, Juntao Zhang, Soumik Sarkar, Adarsh Krishnamurthy, Juan Ren, Anwesha Sarkar

“…We then evaluate the performance of multiple state-of-the-art LLMs on AFM-related queries, offering users flexibility in LLM selection based on their specific needs. …”
Get full text

Supporting energy policy research with large language models: A case study in wind energy siting ordinances by Grant Buster, Pavlo Pinchuk, Jacob Barrons, Ryan McKeever, Aaron Levine, Anthony Lopez

“…A novel contribution of this research is the integration of a decision tree framework with LLMs. Our results show that this approach is 85 to 90 % accurate with outputs that can be used directly in downstream quantitative modeling. …”
Get full text

Student engagement with artificial intelligence tools in academia: a survey of Jordanian universities by Mohammad Al Mashagbeh, Malak Alsharqawi, Uranchimeg Tudevdagva, Hussam J. Khasawneh, Hussam J. Khasawneh

Get full text

Psychology's Questionable Research Fundamentals (QRFs): Key problems in quantitative psychology and psychological measurement beyond Questionable Research Practices (QRPs) by Jana Uher, Jan Ketil Arnulf, Jan Ketil Arnulf, Paul T. Barrett, Moritz Heene, Jörg-Henrik Heine, Jack Martin, Lucas B. Mazur, Lucas B. Mazur, Marek McGann, Robert J. Mislevy, Craig Speelman, Aaro Toomela, Ron Weber, Ron Weber

“…Concluding, just minimising scientific misconduct, as currently believed, and exploiting language-based algorithms (NLP, LLMs) without considering the intricacies of human language will only perpetuate psychology's crises. …”
Get full text

Development of a Comprehensive Evaluation Scale for LLM-Powered Counseling Chatbots (CES-LCC) Using the eDelphi Method by Marco Bolpagni, Silvia Gabrielli

“…<b>Background/Objectives</b>: With advancements in Large Language Models (LLMs), counseling chatbots are becoming essential tools for delivering scalable and accessible mental health support. …”
Get full text

Explicit and Implicit Knowledge in Large-Scale Linguistic Data and Digital Footprints from Social Networks by Maria Pilgun

“…The research results emphasize the importance of integrating various scientific paradigms and artificial intelligence technologies, particularly large language models (LLMs), in the analysis of social networks.…”
Get full text

Impact of large language model (ChatGPT) in healthcare: an umbrella review and evidence synthesis by Usman Iqbal, Afifa Tanweer, Annisa Ristya Rahmanti, David Greenfield, Leon Tsung-Ju Lee, Yu-Chuan Jack Li

Get full text

Anatomy of an AI-powered malicious social botnet by Kai-Cheng Yang, Filippo Menczer

Get full text

Exploring the Impact of Large Language Models on Disease Diagnosis by Ibrahim Almubark

“…The performance evaluation of LLMs involves both qualitative and quantitative measures assessing their diagnostic accuracy. …”
Get full text

Exploring the potential of large language models for integration into an academic statistical consulting service-the EXPOLS study protocol. by Urs Alexander Fichtner, Jochen Knaus, Erika Graf, Georg Koch, Jörg Sahlmann, Dominikus Stelzer, Martin Wolkewitz, Harald Binder, Susanne Weber

“…<h4>Materials and methods</h4>This multimodal study includes four study parts using qualitative and quantitative methods to gather data. Study part (I) is designed as mixed mode study to explore the use of LLMs in supporting statistical consulting and to evaluate the utility, efficiency and satisfaction related to the use of LLMs. …”
Get full text