A Hybrid Transformer Architecture for Multiclass Mental Illness Prediction Using Social Media Text
Mental illness prediction through text involves employing natural language processing (NLP) techniques and deep learning algorithms to analyze textual data for the identification of mental disorders. Therefore, machine learning and deep learning algorithms have been utilized in the existing literatu...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10804794/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832590306324774912 |
|---|---|
| author | Adnan Karamat Muhammad Imran Muhammad Usman Yaseen Rasool Bukhsh Sheraz Aslam Nouman Ashraf |
| author_facet | Adnan Karamat Muhammad Imran Muhammad Usman Yaseen Rasool Bukhsh Sheraz Aslam Nouman Ashraf |
| author_sort | Adnan Karamat |
| collection | DOAJ |
| description | Mental illness prediction through text involves employing natural language processing (NLP) techniques and deep learning algorithms to analyze textual data for the identification of mental disorders. Therefore, machine learning and deep learning algorithms have been utilized in the existing literature for the detection of mental illness. However, current systems exhibit suboptimal performance primarily due to their reliance on traditional embedding techniques and generic language models to generate text embeddings. To address this limitation, there is a requirement for domain-specific pretrained language models that comprehensively understand the context found in posts of person with a psychiatric disability patients. Posts from individuals with mental illness often contain metaphorical expressions, posing a challenge for existing models in understanding such figurative language. In this study, we propose a hybrid transformer architecture, comprising MentalBERT and MelBERT pretrained language models, cascaded with CNN models to generate and concatenate deep features. MentalBERT is pretrained on an extensive corpus of text data specifically related to the mental health domain, while MelBERT is trained on a large corpus of metaphorical data for improved understanding of metaphorical expressions. The results reveal outstanding performance of the proposed architecture with an overall accuracy of 92% and an F1-score of 92%, surpassing state-of-the-art models in comparison. This study underscores the necessity for further research in this field and illustrates the potential of advanced technologies to address mental health issues in contemporary society. |
| format | Article |
| id | doaj-art-0fbbfb9825384e17a4ce56c50e0ac237 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-0fbbfb9825384e17a4ce56c50e0ac2372025-01-24T00:02:00ZengIEEEIEEE Access2169-35362025-01-0113121481216710.1109/ACCESS.2024.351930810804794A Hybrid Transformer Architecture for Multiclass Mental Illness Prediction Using Social Media TextAdnan Karamat0https://orcid.org/0009-0006-5462-1904Muhammad Imran1https://orcid.org/0000-0003-4184-6603Muhammad Usman Yaseen2https://orcid.org/0000-0002-7032-9544Rasool Bukhsh3https://orcid.org/0000-0002-1973-8713Sheraz Aslam4https://orcid.org/0000-0003-4305-0908Nouman Ashraf5https://orcid.org/0000-0002-0164-8031Department of Computer Science, COMSATS University Islamabad (CUI), Islamabad, PakistanDepartment of Computer Science, COMSATS University Islamabad (CUI), Islamabad, PakistanDepartment of Computer Science, COMSATS University Islamabad (CUI), Islamabad, PakistanDepartment of Computer Science, COMSATS University Islamabad (CUI), Islamabad, PakistanDepartment of Electrical Engineering, Computer Engineering and Informatics, Cyprus University of Technology, Limassol, CyprusSchool of Electrical and Electronic Engineering, Technological University Dublin, Dublin, IrelandMental illness prediction through text involves employing natural language processing (NLP) techniques and deep learning algorithms to analyze textual data for the identification of mental disorders. Therefore, machine learning and deep learning algorithms have been utilized in the existing literature for the detection of mental illness. However, current systems exhibit suboptimal performance primarily due to their reliance on traditional embedding techniques and generic language models to generate text embeddings. To address this limitation, there is a requirement for domain-specific pretrained language models that comprehensively understand the context found in posts of person with a psychiatric disability patients. Posts from individuals with mental illness often contain metaphorical expressions, posing a challenge for existing models in understanding such figurative language. In this study, we propose a hybrid transformer architecture, comprising MentalBERT and MelBERT pretrained language models, cascaded with CNN models to generate and concatenate deep features. MentalBERT is pretrained on an extensive corpus of text data specifically related to the mental health domain, while MelBERT is trained on a large corpus of metaphorical data for improved understanding of metaphorical expressions. The results reveal outstanding performance of the proposed architecture with an overall accuracy of 92% and an F1-score of 92%, surpassing state-of-the-art models in comparison. This study underscores the necessity for further research in this field and illustrates the potential of advanced technologies to address mental health issues in contemporary society.https://ieeexplore.ieee.org/document/10804794/Convolutional neural networkdeep learningmental illnessMentalBERTmelBERTsocial media |
| spellingShingle | Adnan Karamat Muhammad Imran Muhammad Usman Yaseen Rasool Bukhsh Sheraz Aslam Nouman Ashraf A Hybrid Transformer Architecture for Multiclass Mental Illness Prediction Using Social Media Text IEEE Access Convolutional neural network deep learning mental illness MentalBERT melBERT social media |
| title | A Hybrid Transformer Architecture for Multiclass Mental Illness Prediction Using Social Media Text |
| title_full | A Hybrid Transformer Architecture for Multiclass Mental Illness Prediction Using Social Media Text |
| title_fullStr | A Hybrid Transformer Architecture for Multiclass Mental Illness Prediction Using Social Media Text |
| title_full_unstemmed | A Hybrid Transformer Architecture for Multiclass Mental Illness Prediction Using Social Media Text |
| title_short | A Hybrid Transformer Architecture for Multiclass Mental Illness Prediction Using Social Media Text |
| title_sort | hybrid transformer architecture for multiclass mental illness prediction using social media text |
| topic | Convolutional neural network deep learning mental illness MentalBERT melBERT social media |
| url | https://ieeexplore.ieee.org/document/10804794/ |
| work_keys_str_mv | AT adnankaramat ahybridtransformerarchitectureformulticlassmentalillnesspredictionusingsocialmediatext AT muhammadimran ahybridtransformerarchitectureformulticlassmentalillnesspredictionusingsocialmediatext AT muhammadusmanyaseen ahybridtransformerarchitectureformulticlassmentalillnesspredictionusingsocialmediatext AT rasoolbukhsh ahybridtransformerarchitectureformulticlassmentalillnesspredictionusingsocialmediatext AT sherazaslam ahybridtransformerarchitectureformulticlassmentalillnesspredictionusingsocialmediatext AT noumanashraf ahybridtransformerarchitectureformulticlassmentalillnesspredictionusingsocialmediatext AT adnankaramat hybridtransformerarchitectureformulticlassmentalillnesspredictionusingsocialmediatext AT muhammadimran hybridtransformerarchitectureformulticlassmentalillnesspredictionusingsocialmediatext AT muhammadusmanyaseen hybridtransformerarchitectureformulticlassmentalillnesspredictionusingsocialmediatext AT rasoolbukhsh hybridtransformerarchitectureformulticlassmentalillnesspredictionusingsocialmediatext AT sherazaslam hybridtransformerarchitectureformulticlassmentalillnesspredictionusingsocialmediatext AT noumanashraf hybridtransformerarchitectureformulticlassmentalillnesspredictionusingsocialmediatext |