QnA Chatbot with Mistral 7B and RAG method: Traffic Law Case Study
Mistral 7B is a language model designed to achieve high efficiency and performance in handling Natural Language Processing (NLP). This research will evaluate the model's effectiveness in legal data processing using the Retrieval-Augmented Generation (RAG) method, focusing on road traffic and tr...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Udayana University, Institute for Research and Community Services
2025-01-01
|
| Series: | Lontar Komputer |
| Online Access: | https://ojs.unud.ac.id/index.php/lontar/article/view/118859 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832575511765712896 |
|---|---|
| author | Muhammad Roiful Anam Agus Subhan Akbar Heru Saputro |
| author_facet | Muhammad Roiful Anam Agus Subhan Akbar Heru Saputro |
| author_sort | Muhammad Roiful Anam |
| collection | DOAJ |
| description | Mistral 7B is a language model designed to achieve high efficiency and performance in handling Natural Language Processing (NLP). This research will evaluate the model's effectiveness in legal data processing using the Retrieval-Augmented Generation (RAG) method, focusing on road traffic and transportation law No 22/2009. The system was built using the LangChain framework, followed by fine-tuning the model and evaluated using BERTScore. Results showed that the fine-tuned Mistral 7B achieved an F1 score of 0.9151, higher than the version without fine-tuning (0.8804) and GPT-4 (0.8364). To improve accuracy, the model utilizes specific keywords that make it easier to find relevant data. Fine-tuning was shown to enhance precision, while the use of key elements in questions helped the model focus more on important information. The results are expected to support the development of artificial intelligence (AI) in Indonesia's legal system and provide practical guidance for applying AI technology in other areas of law. |
| format | Article |
| id | doaj-art-4ef873d3229b4deca87703df3bcc07ce |
| institution | Kabale University |
| issn | 2088-1541 2541-5832 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Udayana University, Institute for Research and Community Services |
| record_format | Article |
| series | Lontar Komputer |
| spelling | doaj-art-4ef873d3229b4deca87703df3bcc07ce2025-01-31T23:56:26ZengUdayana University, Institute for Research and Community ServicesLontar Komputer2088-15412541-58322025-01-01150320721810.24843/LKJITI.2024.v15.i03.p06118859QnA Chatbot with Mistral 7B and RAG method: Traffic Law Case StudyMuhammad Roiful Anam0Agus Subhan AkbarHeru SaputroUniversitas Islam Nahdlatul Ulama JeparaMistral 7B is a language model designed to achieve high efficiency and performance in handling Natural Language Processing (NLP). This research will evaluate the model's effectiveness in legal data processing using the Retrieval-Augmented Generation (RAG) method, focusing on road traffic and transportation law No 22/2009. The system was built using the LangChain framework, followed by fine-tuning the model and evaluated using BERTScore. Results showed that the fine-tuned Mistral 7B achieved an F1 score of 0.9151, higher than the version without fine-tuning (0.8804) and GPT-4 (0.8364). To improve accuracy, the model utilizes specific keywords that make it easier to find relevant data. Fine-tuning was shown to enhance precision, while the use of key elements in questions helped the model focus more on important information. The results are expected to support the development of artificial intelligence (AI) in Indonesia's legal system and provide practical guidance for applying AI technology in other areas of law.https://ojs.unud.ac.id/index.php/lontar/article/view/118859 |
| spellingShingle | Muhammad Roiful Anam Agus Subhan Akbar Heru Saputro QnA Chatbot with Mistral 7B and RAG method: Traffic Law Case Study Lontar Komputer |
| title | QnA Chatbot with Mistral 7B and RAG method: Traffic Law Case Study |
| title_full | QnA Chatbot with Mistral 7B and RAG method: Traffic Law Case Study |
| title_fullStr | QnA Chatbot with Mistral 7B and RAG method: Traffic Law Case Study |
| title_full_unstemmed | QnA Chatbot with Mistral 7B and RAG method: Traffic Law Case Study |
| title_short | QnA Chatbot with Mistral 7B and RAG method: Traffic Law Case Study |
| title_sort | qna chatbot with mistral 7b and rag method traffic law case study |
| url | https://ojs.unud.ac.id/index.php/lontar/article/view/118859 |
| work_keys_str_mv | AT muhammadroifulanam qnachatbotwithmistral7bandragmethodtrafficlawcasestudy AT agussubhanakbar qnachatbotwithmistral7bandragmethodtrafficlawcasestudy AT herusaputro qnachatbotwithmistral7bandragmethodtrafficlawcasestudy |