Enhanced Emotion-Aware Conversational Agent: Analyzing User Behavioral Status for Tailored Reponses in Chatbot Interactions
While chatbots are increasingly popular for communication, their effectiveness is limited by their difficulty in understanding users’ emotions. To address this, this study proposes a new hybrid chatbot model called “TEBC-Net” (Text Emotion Bert CNN Network), which co...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10854433/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832542613917401088 |
|---|---|
| author | S. Abinaya K. S. Ashwin A. Sherly Alphonse |
| author_facet | S. Abinaya K. S. Ashwin A. Sherly Alphonse |
| author_sort | S. Abinaya |
| collection | DOAJ |
| description | While chatbots are increasingly popular for communication, their effectiveness is limited by their difficulty in understanding users’ emotions. To address this, this study proposes a new hybrid chatbot model called “TEBC-Net” (Text Emotion Bert CNN Network), which combines text and video analysis to interpret user emotions and generate more empathetic responses. At the core of TEBC-Net is a multi-modal emotion analysis system. One component uses Bidirectional Encoder Representations from Transformers (BERT), a well-regarded model in natural language processing (NLP), achieving an 87.21% accuracy rate in detecting emotional cues from text inputs. The second component captures users’ facial expressions through webcam footage. It begins by detecting faces using a pre-trained classifier like Haarcascade. Then, to improve emotion recognition, it preprocesses the image through brightness adjustments and contrast enhancement with Automatic CLAHE and dual gamma correction. This processed image is analyzed by a Convolutional Neural Network (CNN) model trained specifically for emotion recognition, reaching 74.14% accuracy by assigning probabilities to different emotions. By integrating insights from both text and video analysis, TEBC-Net gains a comprehensive understanding of the user’s emotional state and intent. This combined data then informs the chatbot’s response generation module, enabling it to craft responses that are both empathetic and more directly aligned with the user’s emotional needs. |
| format | Article |
| id | doaj-art-d80721b5ecb549158e658332bc1bbee5 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-d80721b5ecb549158e658332bc1bbee52025-02-04T00:00:42ZengIEEEIEEE Access2169-35362025-01-0113197701978710.1109/ACCESS.2025.353419710854433Enhanced Emotion-Aware Conversational Agent: Analyzing User Behavioral Status for Tailored Reponses in Chatbot InteractionsS. Abinaya0https://orcid.org/0000-0001-7957-7934K. S. Ashwin1https://orcid.org/0009-0008-3525-287XA. Sherly Alphonse2https://orcid.org/0000-0002-0019-9940School of Computer Science and Engineering, Vellore Institute of Technology, Chennai, IndiaSchool of Computer Science and Engineering, Vellore Institute of Technology, Chennai, IndiaSchool of Computer Science and Engineering, Vellore Institute of Technology, Chennai, IndiaWhile chatbots are increasingly popular for communication, their effectiveness is limited by their difficulty in understanding users’ emotions. To address this, this study proposes a new hybrid chatbot model called “TEBC-Net” (Text Emotion Bert CNN Network), which combines text and video analysis to interpret user emotions and generate more empathetic responses. At the core of TEBC-Net is a multi-modal emotion analysis system. One component uses Bidirectional Encoder Representations from Transformers (BERT), a well-regarded model in natural language processing (NLP), achieving an 87.21% accuracy rate in detecting emotional cues from text inputs. The second component captures users’ facial expressions through webcam footage. It begins by detecting faces using a pre-trained classifier like Haarcascade. Then, to improve emotion recognition, it preprocesses the image through brightness adjustments and contrast enhancement with Automatic CLAHE and dual gamma correction. This processed image is analyzed by a Convolutional Neural Network (CNN) model trained specifically for emotion recognition, reaching 74.14% accuracy by assigning probabilities to different emotions. By integrating insights from both text and video analysis, TEBC-Net gains a comprehensive understanding of the user’s emotional state and intent. This combined data then informs the chatbot’s response generation module, enabling it to craft responses that are both empathetic and more directly aligned with the user’s emotional needs.https://ieeexplore.ieee.org/document/10854433/Chatbotbidirectional encoder representations from transformersnatural language processingemotion recognitionautomatic CLAHE with dual gamma correctionconvolutional neural network |
| spellingShingle | S. Abinaya K. S. Ashwin A. Sherly Alphonse Enhanced Emotion-Aware Conversational Agent: Analyzing User Behavioral Status for Tailored Reponses in Chatbot Interactions IEEE Access Chatbot bidirectional encoder representations from transformers natural language processing emotion recognition automatic CLAHE with dual gamma correction convolutional neural network |
| title | Enhanced Emotion-Aware Conversational Agent: Analyzing User Behavioral Status for Tailored Reponses in Chatbot Interactions |
| title_full | Enhanced Emotion-Aware Conversational Agent: Analyzing User Behavioral Status for Tailored Reponses in Chatbot Interactions |
| title_fullStr | Enhanced Emotion-Aware Conversational Agent: Analyzing User Behavioral Status for Tailored Reponses in Chatbot Interactions |
| title_full_unstemmed | Enhanced Emotion-Aware Conversational Agent: Analyzing User Behavioral Status for Tailored Reponses in Chatbot Interactions |
| title_short | Enhanced Emotion-Aware Conversational Agent: Analyzing User Behavioral Status for Tailored Reponses in Chatbot Interactions |
| title_sort | enhanced emotion aware conversational agent analyzing user behavioral status for tailored reponses in chatbot interactions |
| topic | Chatbot bidirectional encoder representations from transformers natural language processing emotion recognition automatic CLAHE with dual gamma correction convolutional neural network |
| url | https://ieeexplore.ieee.org/document/10854433/ |
| work_keys_str_mv | AT sabinaya enhancedemotionawareconversationalagentanalyzinguserbehavioralstatusfortailoredreponsesinchatbotinteractions AT ksashwin enhancedemotionawareconversationalagentanalyzinguserbehavioralstatusfortailoredreponsesinchatbotinteractions AT asherlyalphonse enhancedemotionawareconversationalagentanalyzinguserbehavioralstatusfortailoredreponsesinchatbotinteractions |