Enhancing classification efficiency in capsule networks through windowed routing: tackling gradient vanishing, dynamic routing, and computational complexity challenges
Abstract Capsule networks overcome the two drawbacks of convolutional neural networks: weak rotated object recognition and poor spatial discrimination. However, they still have encountered problems with complex images, including high computational cost and limited accuracy. To address these challeng...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Springer
2024-11-01
|
| Series: | Complex & Intelligent Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1007/s40747-024-01640-8 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832571150111080448 |
|---|---|
| author | Gangqi Chen Zhaoyong Mao Junge Shen Dongdong Hou |
| author_facet | Gangqi Chen Zhaoyong Mao Junge Shen Dongdong Hou |
| author_sort | Gangqi Chen |
| collection | DOAJ |
| description | Abstract Capsule networks overcome the two drawbacks of convolutional neural networks: weak rotated object recognition and poor spatial discrimination. However, they still have encountered problems with complex images, including high computational cost and limited accuracy. To address these challenges, this work has developed effective solutions. Specifically, a novel windowed dynamic up-and-down attention routing process is first introduced, which can effectively reduce the computational complexity from quadratic to linear order. A novel deconvolution-based decoder is also used to further reduce the computational complexity. Then, a novel LayerNorm strategy is used to pre-process neuron values in the squash function. This prevents saturation and mitigates the gradient vanishing problem. In addition, a novel gradient-friendly network structure is developed to facilitate the extraction of complex features with deeper networks. Experiments show that our methods are effective and competitive, outperforming existing techniques. |
| format | Article |
| id | doaj-art-2505e07d61fc4156984424aa72b80f6a |
| institution | Kabale University |
| issn | 2199-4536 2198-6053 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | Springer |
| record_format | Article |
| series | Complex & Intelligent Systems |
| spelling | doaj-art-2505e07d61fc4156984424aa72b80f6a2025-02-02T12:49:48ZengSpringerComplex & Intelligent Systems2199-45362198-60532024-11-0111111610.1007/s40747-024-01640-8Enhancing classification efficiency in capsule networks through windowed routing: tackling gradient vanishing, dynamic routing, and computational complexity challengesGangqi Chen0Zhaoyong Mao1Junge Shen2Dongdong Hou3School of Marine Science and Technology, Northwestern Polytechnical UniversityUnmanned System Research Institute, Northwestern Polytechnical UniversityUnmanned System Research Institute, Northwestern Polytechnical University713th Research Institute, China State Ship building Co. LtdAbstract Capsule networks overcome the two drawbacks of convolutional neural networks: weak rotated object recognition and poor spatial discrimination. However, they still have encountered problems with complex images, including high computational cost and limited accuracy. To address these challenges, this work has developed effective solutions. Specifically, a novel windowed dynamic up-and-down attention routing process is first introduced, which can effectively reduce the computational complexity from quadratic to linear order. A novel deconvolution-based decoder is also used to further reduce the computational complexity. Then, a novel LayerNorm strategy is used to pre-process neuron values in the squash function. This prevents saturation and mitigates the gradient vanishing problem. In addition, a novel gradient-friendly network structure is developed to facilitate the extraction of complex features with deeper networks. Experiments show that our methods are effective and competitive, outperforming existing techniques.https://doi.org/10.1007/s40747-024-01640-8Capsule NetworkWindow AttentionGradient Vanishing ProblemImage Classification |
| spellingShingle | Gangqi Chen Zhaoyong Mao Junge Shen Dongdong Hou Enhancing classification efficiency in capsule networks through windowed routing: tackling gradient vanishing, dynamic routing, and computational complexity challenges Complex & Intelligent Systems Capsule Network Window Attention Gradient Vanishing Problem Image Classification |
| title | Enhancing classification efficiency in capsule networks through windowed routing: tackling gradient vanishing, dynamic routing, and computational complexity challenges |
| title_full | Enhancing classification efficiency in capsule networks through windowed routing: tackling gradient vanishing, dynamic routing, and computational complexity challenges |
| title_fullStr | Enhancing classification efficiency in capsule networks through windowed routing: tackling gradient vanishing, dynamic routing, and computational complexity challenges |
| title_full_unstemmed | Enhancing classification efficiency in capsule networks through windowed routing: tackling gradient vanishing, dynamic routing, and computational complexity challenges |
| title_short | Enhancing classification efficiency in capsule networks through windowed routing: tackling gradient vanishing, dynamic routing, and computational complexity challenges |
| title_sort | enhancing classification efficiency in capsule networks through windowed routing tackling gradient vanishing dynamic routing and computational complexity challenges |
| topic | Capsule Network Window Attention Gradient Vanishing Problem Image Classification |
| url | https://doi.org/10.1007/s40747-024-01640-8 |
| work_keys_str_mv | AT gangqichen enhancingclassificationefficiencyincapsulenetworksthroughwindowedroutingtacklinggradientvanishingdynamicroutingandcomputationalcomplexitychallenges AT zhaoyongmao enhancingclassificationefficiencyincapsulenetworksthroughwindowedroutingtacklinggradientvanishingdynamicroutingandcomputationalcomplexitychallenges AT jungeshen enhancingclassificationefficiencyincapsulenetworksthroughwindowedroutingtacklinggradientvanishingdynamicroutingandcomputationalcomplexitychallenges AT dongdonghou enhancingclassificationefficiencyincapsulenetworksthroughwindowedroutingtacklinggradientvanishingdynamicroutingandcomputationalcomplexitychallenges |