Arch-Eval benchmark for assessing chinese architectural domain knowledge in large language models

Arch-Eval benchmark for assessing chinese architectural domain knowledge in large language models

Abstract The burgeoning application of Large Language Models (LLMs) in Natural Language Processing (NLP) has prompted scrutiny of their domain-specific knowledge processing, especially in the construction industry. Despite high demand, there is a scarcity of evaluative studies for LLMs in this area....

Full description

Saved in:
Bibliographic Details
Main Authors: Jie Wu, Mincheng Jiang, Juntian Fan, Shimin Li, Hongtao Xu, Ye Zhao
Format: Article
Language:English
Published: Nature Portfolio 2025-04-01
Series:Scientific Reports
Subjects:
LLMs’ assessment
Construction knowledge
Domain specialization
Answer-only (AO) evaluation
Chain-of-thought (COT)
Online Access:https://doi.org/10.1038/s41598-025-98236-0
Tags: Add Tag
No Tags, Be the first to tag this record!

Internet

https://doi.org/10.1038/s41598-025-98236-0

Similar Items