Quantum theory-inspired inter-sentence semantic interaction model for textual adversarial defense
Abstract Deep neural networks have a recognized susceptibility to diverse forms of adversarial attacks in the field of natural language processing and such a security issue poses substantial security risks and erodes trust in artificial intelligence applications among people who use them. Meanwhile,...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Springer
2024-12-01
|
| Series: | Complex & Intelligent Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1007/s40747-024-01733-4 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832571148577013760 |
|---|---|
| author | Jiacheng Huang Long Chen Xiaoyin Yi Ning Yu |
| author_facet | Jiacheng Huang Long Chen Xiaoyin Yi Ning Yu |
| author_sort | Jiacheng Huang |
| collection | DOAJ |
| description | Abstract Deep neural networks have a recognized susceptibility to diverse forms of adversarial attacks in the field of natural language processing and such a security issue poses substantial security risks and erodes trust in artificial intelligence applications among people who use them. Meanwhile, quantum theory-inspired models that represent word composition as a quantum mixture of words have modeled the non-linear semantic interaction. However, modeling without considering the non-linear semantic interaction between sentences in the current literature does not exploit the potential of the quantum probabilistic description for improving the robustness in adversarial settings. In the present study, a novel quantum theory-inspired inter-sentence semantic interaction model is proposed for enhancing adversarial robustness via fusing contextual semantics. More specifically, it is analyzed why humans are able to understand textual adversarial examples, and a crucial point is observed that humans are adept at associating information from the context to comprehend a paragraph. Guided by this insight, the input text is segmented into subsentences, with the model simulating contextual comprehension by representing each subsentence as a particle within a mixture system, utilizing a density matrix to model inter-sentence interactions. A loss function integrating cross-entropy and orthogonality losses is employed to encourage the orthogonality of measurement states. Comprehensive experiments are conducted to validate the efficacy of proposed methodology, and the results underscore its superiority over baseline models even commercial applications based on large language models in terms of accuracy across diverse adversarial attack scenarios, showing the potential of proposed approach in enhancing the robustness of neural networks under adversarial attacks. |
| format | Article |
| id | doaj-art-1e0137fb2f514397bdbd1746dafef019 |
| institution | Kabale University |
| issn | 2199-4536 2198-6053 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Springer |
| record_format | Article |
| series | Complex & Intelligent Systems |
| spelling | doaj-art-1e0137fb2f514397bdbd1746dafef0192025-02-02T12:49:41ZengSpringerComplex & Intelligent Systems2199-45362198-60532024-12-0111111610.1007/s40747-024-01733-4Quantum theory-inspired inter-sentence semantic interaction model for textual adversarial defenseJiacheng Huang0Long Chen1Xiaoyin Yi2Ning Yu3School of Computer Science and Technology, Chongqing University of Posts and TelecommunicationsSchool of Computer Science and Technology, Chongqing University of Posts and TelecommunicationsSchool of Computer Science and Technology, Chongqing University of Posts and TelecommunicationsSchool of Computer Science and Technology, Chongqing University of Posts and TelecommunicationsAbstract Deep neural networks have a recognized susceptibility to diverse forms of adversarial attacks in the field of natural language processing and such a security issue poses substantial security risks and erodes trust in artificial intelligence applications among people who use them. Meanwhile, quantum theory-inspired models that represent word composition as a quantum mixture of words have modeled the non-linear semantic interaction. However, modeling without considering the non-linear semantic interaction between sentences in the current literature does not exploit the potential of the quantum probabilistic description for improving the robustness in adversarial settings. In the present study, a novel quantum theory-inspired inter-sentence semantic interaction model is proposed for enhancing adversarial robustness via fusing contextual semantics. More specifically, it is analyzed why humans are able to understand textual adversarial examples, and a crucial point is observed that humans are adept at associating information from the context to comprehend a paragraph. Guided by this insight, the input text is segmented into subsentences, with the model simulating contextual comprehension by representing each subsentence as a particle within a mixture system, utilizing a density matrix to model inter-sentence interactions. A loss function integrating cross-entropy and orthogonality losses is employed to encourage the orthogonality of measurement states. Comprehensive experiments are conducted to validate the efficacy of proposed methodology, and the results underscore its superiority over baseline models even commercial applications based on large language models in terms of accuracy across diverse adversarial attack scenarios, showing the potential of proposed approach in enhancing the robustness of neural networks under adversarial attacks.https://doi.org/10.1007/s40747-024-01733-4Quantum probabilityNatural language processingAdversarial examplesInter-sentence semantic interactions |
| spellingShingle | Jiacheng Huang Long Chen Xiaoyin Yi Ning Yu Quantum theory-inspired inter-sentence semantic interaction model for textual adversarial defense Complex & Intelligent Systems Quantum probability Natural language processing Adversarial examples Inter-sentence semantic interactions |
| title | Quantum theory-inspired inter-sentence semantic interaction model for textual adversarial defense |
| title_full | Quantum theory-inspired inter-sentence semantic interaction model for textual adversarial defense |
| title_fullStr | Quantum theory-inspired inter-sentence semantic interaction model for textual adversarial defense |
| title_full_unstemmed | Quantum theory-inspired inter-sentence semantic interaction model for textual adversarial defense |
| title_short | Quantum theory-inspired inter-sentence semantic interaction model for textual adversarial defense |
| title_sort | quantum theory inspired inter sentence semantic interaction model for textual adversarial defense |
| topic | Quantum probability Natural language processing Adversarial examples Inter-sentence semantic interactions |
| url | https://doi.org/10.1007/s40747-024-01733-4 |
| work_keys_str_mv | AT jiachenghuang quantumtheoryinspiredintersentencesemanticinteractionmodelfortextualadversarialdefense AT longchen quantumtheoryinspiredintersentencesemanticinteractionmodelfortextualadversarialdefense AT xiaoyinyi quantumtheoryinspiredintersentencesemanticinteractionmodelfortextualadversarialdefense AT ningyu quantumtheoryinspiredintersentencesemanticinteractionmodelfortextualadversarialdefense |