An Innovative Solution to Design Problems: Applying the Chain-of-Thought Technique to Integrate LLM-Based Agents With Concept Generation Methods
To enhance the application capabilities of large language models (LLMs) in conceptual design, this study explores how to achieve deep integration between LLM-based agents and concept generation methods using the chain-of-thought (CoT) technique and evaluates its feasibility. Using GPT-4 as a case st...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10747324/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832590306776711168 |
|---|---|
| author | Shijun Ge Yuanbo Sun Yin Cui Dapeng Wei |
| author_facet | Shijun Ge Yuanbo Sun Yin Cui Dapeng Wei |
| author_sort | Shijun Ge |
| collection | DOAJ |
| description | To enhance the application capabilities of large language models (LLMs) in conceptual design, this study explores how to achieve deep integration between LLM-based agents and concept generation methods using the chain-of-thought (CoT) technique and evaluates its feasibility. Using GPT-4 as a case study, we designed two agents: IntelliStorm (based on the unstructured brainstorming method) and EvoluTRIZ (based on the structured TRIZ method). Thirty participants were recruited, and through two experimental phases spaced one month apart, a comparative analysis of the effects of collaboration groups (human-agent vs. human-human) and concept generation methods (brainstorming vs. TRIZ) on participants’ physiological activation and creative thinking performance were conducted. The results show that the involvement of LLM-based agents can effectively reduce participants’ electrodermal activity(EDA) response levels, indicating a reduction in cognitive load. Moreover, participants maintained their distinct physiological patterns and performance advantages across different concept generation methods. For example, IntelliStorm, like brainstorming, evokes stronger responses to information stimuli, demonstrating superior thinking fluency; EvoluTRIZ, like the TRIZ, exhibits a higher frequency of information responses, showcasing enhanced thinking elaboration. However, originality tends to favor human-human collaboration. The findings confirm that integrating LLMs with traditional concept generation methods is an effective strategy made possible by combining CoT and retrieval-augmented generation (RAG) technologies. In the future, LLM-based agents are expected to achieve broader application in the design field by incorporating additional concept generation methods. |
| format | Article |
| id | doaj-art-107653736c934f0291dc763e0c738712 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-107653736c934f0291dc763e0c7387122025-01-24T00:01:58ZengIEEEIEEE Access2169-35362025-01-0113104991051210.1109/ACCESS.2024.349405410747324An Innovative Solution to Design Problems: Applying the Chain-of-Thought Technique to Integrate LLM-Based Agents With Concept Generation MethodsShijun Ge0https://orcid.org/0009-0007-4380-5836Yuanbo Sun1Yin Cui2https://orcid.org/0009-0003-4134-8687Dapeng Wei3School of Design and Art, Beijing Institute of Technology, Beijing, ChinaSchool of Design and Art, Beijing Institute of Technology, Beijing, ChinaSchool of Design and Innovation, Shenzhen Technology University, Shenzhen, ChinaSchool of Design and Art, Beijing Institute of Technology, Beijing, ChinaTo enhance the application capabilities of large language models (LLMs) in conceptual design, this study explores how to achieve deep integration between LLM-based agents and concept generation methods using the chain-of-thought (CoT) technique and evaluates its feasibility. Using GPT-4 as a case study, we designed two agents: IntelliStorm (based on the unstructured brainstorming method) and EvoluTRIZ (based on the structured TRIZ method). Thirty participants were recruited, and through two experimental phases spaced one month apart, a comparative analysis of the effects of collaboration groups (human-agent vs. human-human) and concept generation methods (brainstorming vs. TRIZ) on participants’ physiological activation and creative thinking performance were conducted. The results show that the involvement of LLM-based agents can effectively reduce participants’ electrodermal activity(EDA) response levels, indicating a reduction in cognitive load. Moreover, participants maintained their distinct physiological patterns and performance advantages across different concept generation methods. For example, IntelliStorm, like brainstorming, evokes stronger responses to information stimuli, demonstrating superior thinking fluency; EvoluTRIZ, like the TRIZ, exhibits a higher frequency of information responses, showcasing enhanced thinking elaboration. However, originality tends to favor human-human collaboration. The findings confirm that integrating LLMs with traditional concept generation methods is an effective strategy made possible by combining CoT and retrieval-augmented generation (RAG) technologies. In the future, LLM-based agents are expected to achieve broader application in the design field by incorporating additional concept generation methods.https://ieeexplore.ieee.org/document/10747324/LLM-based agentchain-of-thought fine-tuningconcept generation methodEDAhuman-agent collaborationhuman-human collaboration |
| spellingShingle | Shijun Ge Yuanbo Sun Yin Cui Dapeng Wei An Innovative Solution to Design Problems: Applying the Chain-of-Thought Technique to Integrate LLM-Based Agents With Concept Generation Methods IEEE Access LLM-based agent chain-of-thought fine-tuning concept generation method EDA human-agent collaboration human-human collaboration |
| title | An Innovative Solution to Design Problems: Applying the Chain-of-Thought Technique to Integrate LLM-Based Agents With Concept Generation Methods |
| title_full | An Innovative Solution to Design Problems: Applying the Chain-of-Thought Technique to Integrate LLM-Based Agents With Concept Generation Methods |
| title_fullStr | An Innovative Solution to Design Problems: Applying the Chain-of-Thought Technique to Integrate LLM-Based Agents With Concept Generation Methods |
| title_full_unstemmed | An Innovative Solution to Design Problems: Applying the Chain-of-Thought Technique to Integrate LLM-Based Agents With Concept Generation Methods |
| title_short | An Innovative Solution to Design Problems: Applying the Chain-of-Thought Technique to Integrate LLM-Based Agents With Concept Generation Methods |
| title_sort | innovative solution to design problems applying the chain of thought technique to integrate llm based agents with concept generation methods |
| topic | LLM-based agent chain-of-thought fine-tuning concept generation method EDA human-agent collaboration human-human collaboration |
| url | https://ieeexplore.ieee.org/document/10747324/ |
| work_keys_str_mv | AT shijunge aninnovativesolutiontodesignproblemsapplyingthechainofthoughttechniquetointegratellmbasedagentswithconceptgenerationmethods AT yuanbosun aninnovativesolutiontodesignproblemsapplyingthechainofthoughttechniquetointegratellmbasedagentswithconceptgenerationmethods AT yincui aninnovativesolutiontodesignproblemsapplyingthechainofthoughttechniquetointegratellmbasedagentswithconceptgenerationmethods AT dapengwei aninnovativesolutiontodesignproblemsapplyingthechainofthoughttechniquetointegratellmbasedagentswithconceptgenerationmethods AT shijunge innovativesolutiontodesignproblemsapplyingthechainofthoughttechniquetointegratellmbasedagentswithconceptgenerationmethods AT yuanbosun innovativesolutiontodesignproblemsapplyingthechainofthoughttechniquetointegratellmbasedagentswithconceptgenerationmethods AT yincui innovativesolutiontodesignproblemsapplyingthechainofthoughttechniquetointegratellmbasedagentswithconceptgenerationmethods AT dapengwei innovativesolutiontodesignproblemsapplyingthechainofthoughttechniquetointegratellmbasedagentswithconceptgenerationmethods |