A geometric approach for accelerating neural networks designed for classification problems
Abstract This paper proposes a geometric-based technique for compressing convolutional neural networks to accelerate computations and improve generalization by eliminating non-informative components. The technique utilizes a geometric index called separation index to evaluate the functionality of ne...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2024-07-01
|
| Series: | Scientific Reports |
| Subjects: | |
| Online Access: | https://doi.org/10.1038/s41598-024-68172-6 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832585684906409984 |
|---|---|
| author | Mohsen Saffar Ahmad Kalhor Ali Habibnia |
| author_facet | Mohsen Saffar Ahmad Kalhor Ali Habibnia |
| author_sort | Mohsen Saffar |
| collection | DOAJ |
| description | Abstract This paper proposes a geometric-based technique for compressing convolutional neural networks to accelerate computations and improve generalization by eliminating non-informative components. The technique utilizes a geometric index called separation index to evaluate the functionality of network elements such as layers and filters. By applying this index along with center-based separation index, a systematic algorithm is proposed that optimally compresses convolutional and fully connected layers. The algorithm excludes layers with low performance, selects the best subset of filters in the filtering layers, and tunes the parameters of fully connected layers using center-based separation index. An illustrative example of classifying CIFAR-10 dataset is presented to explain the algorithm step-by-step. The proposed method achieves impressive pruning results on networks trained by CIFAR-10 and ImageNet datasets, with 87.5%, 77.6%, and 78.8% of VGG16, GoogLeNet, and DenseNet parameters pruned, respectively. Comparisons with state-of-the-art works are provided to demonstrate the effectiveness of the proposed method. |
| format | Article |
| id | doaj-art-0c04483c62b142d5b1b8fc5194aa6671 |
| institution | Kabale University |
| issn | 2045-2322 |
| language | English |
| publishDate | 2024-07-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-0c04483c62b142d5b1b8fc5194aa66712025-01-26T12:35:09ZengNature PortfolioScientific Reports2045-23222024-07-0114111610.1038/s41598-024-68172-6A geometric approach for accelerating neural networks designed for classification problemsMohsen Saffar0Ahmad Kalhor1Ali Habibnia2School of Electrical and Computer Engineering, College of Engineering, University of TehranSchool of Electrical and Computer Engineering, College of Engineering, University of TehranDepartment of Economics and the Computational Modeling and Data Analytics, College of Science, Virginia Polytechnic Institute and State UniversityAbstract This paper proposes a geometric-based technique for compressing convolutional neural networks to accelerate computations and improve generalization by eliminating non-informative components. The technique utilizes a geometric index called separation index to evaluate the functionality of network elements such as layers and filters. By applying this index along with center-based separation index, a systematic algorithm is proposed that optimally compresses convolutional and fully connected layers. The algorithm excludes layers with low performance, selects the best subset of filters in the filtering layers, and tunes the parameters of fully connected layers using center-based separation index. An illustrative example of classifying CIFAR-10 dataset is presented to explain the algorithm step-by-step. The proposed method achieves impressive pruning results on networks trained by CIFAR-10 and ImageNet datasets, with 87.5%, 77.6%, and 78.8% of VGG16, GoogLeNet, and DenseNet parameters pruned, respectively. Comparisons with state-of-the-art works are provided to demonstrate the effectiveness of the proposed method.https://doi.org/10.1038/s41598-024-68172-6Network compressionConvolutional neural networkDataflow evaluationSeparation index |
| spellingShingle | Mohsen Saffar Ahmad Kalhor Ali Habibnia A geometric approach for accelerating neural networks designed for classification problems Scientific Reports Network compression Convolutional neural network Dataflow evaluation Separation index |
| title | A geometric approach for accelerating neural networks designed for classification problems |
| title_full | A geometric approach for accelerating neural networks designed for classification problems |
| title_fullStr | A geometric approach for accelerating neural networks designed for classification problems |
| title_full_unstemmed | A geometric approach for accelerating neural networks designed for classification problems |
| title_short | A geometric approach for accelerating neural networks designed for classification problems |
| title_sort | geometric approach for accelerating neural networks designed for classification problems |
| topic | Network compression Convolutional neural network Dataflow evaluation Separation index |
| url | https://doi.org/10.1038/s41598-024-68172-6 |
| work_keys_str_mv | AT mohsensaffar ageometricapproachforacceleratingneuralnetworksdesignedforclassificationproblems AT ahmadkalhor ageometricapproachforacceleratingneuralnetworksdesignedforclassificationproblems AT alihabibnia ageometricapproachforacceleratingneuralnetworksdesignedforclassificationproblems AT mohsensaffar geometricapproachforacceleratingneuralnetworksdesignedforclassificationproblems AT ahmadkalhor geometricapproachforacceleratingneuralnetworksdesignedforclassificationproblems AT alihabibnia geometricapproachforacceleratingneuralnetworksdesignedforclassificationproblems |