Multiscale Time-Frequency Sparse Transformer Based on Partly Interpretable Method for Bearing Fault Diagnosis
Transformer model is being gradually studied and applied in bearing fault diagnosis tasks, which can overcome the feature extraction defects caused by long-term dependencies in convolution neural network (CNN) and recurrent neural network (RNN). To optimize the structure of existing transformer-like...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2023-01-01
|
| Series: | Shock and Vibration |
| Online Access: | http://dx.doi.org/10.1155/2023/1639287 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832555353960611840 |
|---|---|
| author | Shouquan Che Jianfeng Lu Congwang Bao Caihong Zhang Yongzhi Liu |
| author_facet | Shouquan Che Jianfeng Lu Congwang Bao Caihong Zhang Yongzhi Liu |
| author_sort | Shouquan Che |
| collection | DOAJ |
| description | Transformer model is being gradually studied and applied in bearing fault diagnosis tasks, which can overcome the feature extraction defects caused by long-term dependencies in convolution neural network (CNN) and recurrent neural network (RNN). To optimize the structure of existing transformer-like methods and improve the diagnostic accuracy, we proposed a novel method based on the multiscale time-frequency sparse transformer (MTFST) in this paper. First, a novel tokenizer based on shot-time Fourier transform (STFT) is designed, which processes the 1D format raw signals into 2D format discrete time-frequency sequences in the embedding space. Second, a sparse self-attention mechanism is designed to eliminate the feature mapping defect in naive self-attention mechanism. Then, the novel encoder-decoder structure is presented, the multiple encoders are employed to extract the hidden feature of different time-frequency sequences obtained by STFT with different window widths, and the decoder is used to remap the deep information and connect to the classifier for discriminating fault types. The proposed method is tested in the XJTU-SY bearing dataset and self-made experiment rig dataset, and the following work is conducted. The influences of hyperparameters on diagnosis accuracy and number of parameters are analysed in detail. The weights of the attention mechanism (AM) are visualized and analysed to study the interpretability, which explains the partly working pattern of the network. In the comparison test with other existing CNN, RNN, and transformer models, the diagnosis accuracy of different methods is statistically analysed, feature vectors are presented via the t-distributed stochastic neighbor embedding (t-SNE) method, and the proposed MTFST obtains the best accuracy and feature distribution form. The results demonstrate the effectiveness and superiority of the proposed method in bearing fault diagnosis. |
| format | Article |
| id | doaj-art-b7ad4d6e6e9f466f8815938873f7a84b |
| institution | Kabale University |
| issn | 1875-9203 |
| language | English |
| publishDate | 2023-01-01 |
| publisher | Wiley |
| record_format | Article |
| series | Shock and Vibration |
| spelling | doaj-art-b7ad4d6e6e9f466f8815938873f7a84b2025-02-03T05:48:29ZengWileyShock and Vibration1875-92032023-01-01202310.1155/2023/1639287Multiscale Time-Frequency Sparse Transformer Based on Partly Interpretable Method for Bearing Fault DiagnosisShouquan Che0Jianfeng Lu1Congwang Bao2Caihong Zhang3Yongzhi Liu4College of Mining and Mechanical EngineeringCollege of Mechanical EngineeringCollege of Mining and Mechanical EngineeringCollege of Mining and Mechanical EngineeringCollege of Mining and Mechanical EngineeringTransformer model is being gradually studied and applied in bearing fault diagnosis tasks, which can overcome the feature extraction defects caused by long-term dependencies in convolution neural network (CNN) and recurrent neural network (RNN). To optimize the structure of existing transformer-like methods and improve the diagnostic accuracy, we proposed a novel method based on the multiscale time-frequency sparse transformer (MTFST) in this paper. First, a novel tokenizer based on shot-time Fourier transform (STFT) is designed, which processes the 1D format raw signals into 2D format discrete time-frequency sequences in the embedding space. Second, a sparse self-attention mechanism is designed to eliminate the feature mapping defect in naive self-attention mechanism. Then, the novel encoder-decoder structure is presented, the multiple encoders are employed to extract the hidden feature of different time-frequency sequences obtained by STFT with different window widths, and the decoder is used to remap the deep information and connect to the classifier for discriminating fault types. The proposed method is tested in the XJTU-SY bearing dataset and self-made experiment rig dataset, and the following work is conducted. The influences of hyperparameters on diagnosis accuracy and number of parameters are analysed in detail. The weights of the attention mechanism (AM) are visualized and analysed to study the interpretability, which explains the partly working pattern of the network. In the comparison test with other existing CNN, RNN, and transformer models, the diagnosis accuracy of different methods is statistically analysed, feature vectors are presented via the t-distributed stochastic neighbor embedding (t-SNE) method, and the proposed MTFST obtains the best accuracy and feature distribution form. The results demonstrate the effectiveness and superiority of the proposed method in bearing fault diagnosis.http://dx.doi.org/10.1155/2023/1639287 |
| spellingShingle | Shouquan Che Jianfeng Lu Congwang Bao Caihong Zhang Yongzhi Liu Multiscale Time-Frequency Sparse Transformer Based on Partly Interpretable Method for Bearing Fault Diagnosis Shock and Vibration |
| title | Multiscale Time-Frequency Sparse Transformer Based on Partly Interpretable Method for Bearing Fault Diagnosis |
| title_full | Multiscale Time-Frequency Sparse Transformer Based on Partly Interpretable Method for Bearing Fault Diagnosis |
| title_fullStr | Multiscale Time-Frequency Sparse Transformer Based on Partly Interpretable Method for Bearing Fault Diagnosis |
| title_full_unstemmed | Multiscale Time-Frequency Sparse Transformer Based on Partly Interpretable Method for Bearing Fault Diagnosis |
| title_short | Multiscale Time-Frequency Sparse Transformer Based on Partly Interpretable Method for Bearing Fault Diagnosis |
| title_sort | multiscale time frequency sparse transformer based on partly interpretable method for bearing fault diagnosis |
| url | http://dx.doi.org/10.1155/2023/1639287 |
| work_keys_str_mv | AT shouquanche multiscaletimefrequencysparsetransformerbasedonpartlyinterpretablemethodforbearingfaultdiagnosis AT jianfenglu multiscaletimefrequencysparsetransformerbasedonpartlyinterpretablemethodforbearingfaultdiagnosis AT congwangbao multiscaletimefrequencysparsetransformerbasedonpartlyinterpretablemethodforbearingfaultdiagnosis AT caihongzhang multiscaletimefrequencysparsetransformerbasedonpartlyinterpretablemethodforbearingfaultdiagnosis AT yongzhiliu multiscaletimefrequencysparsetransformerbasedonpartlyinterpretablemethodforbearingfaultdiagnosis |