Assessing bias and computational efficiency in vision transformers using early exits
Abstract Face recognition with deep learning is generally approached as a problem of capacity. The field has seen progressively deeper, more complex models or larger, more highly variant data sets. The data sets can be problematic, as they are often scraped indiscriminately from the internet. This r...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2025-01-01
|
| Series: | EURASIP Journal on Image and Video Processing |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s13640-024-00658-9 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832585458477957120 |
|---|---|
| author | Seth Nixon Pietro Ruiu Marinella Cadoni Andrea Lagorio Massimo Tistarelli |
| author_facet | Seth Nixon Pietro Ruiu Marinella Cadoni Andrea Lagorio Massimo Tistarelli |
| author_sort | Seth Nixon |
| collection | DOAJ |
| description | Abstract Face recognition with deep learning is generally approached as a problem of capacity. The field has seen progressively deeper, more complex models or larger, more highly variant data sets. The data sets can be problematic, as they are often scraped indiscriminately from the internet. This results in an uncertain, and often heavily unbalanced distribution of race, gender, age and other aspects of the subjects, which is then manifested in the decisions of the models trained on them. The carbon footprint of machine learning is a concern. A real push is developing to reduce the energy consumption of machine learning as we strive for a more eco-friendly society. In addition, due to many instances of misuse by law enforcement and other agencies, unbiased models for face recognition are now fundamental to the practical application of the field. We present an approach using the state of the art Vision Transformer and Early Exits for reducing compute budget without significantly affecting performance. We develop a system for face recognition and identification with a closed-set gallery and show that with a small reduction in performance, a reasonable reduction in compute cost can be obtained using our method. Second, we investigate how these early Exits interact with the bias model through a robust evaluation of matching scores on a racially balanced data set. We show that matching scores vary heavily between cohorts, and these variations are magnified at the earlier exits. |
| format | Article |
| id | doaj-art-ddf28052d0ed4730afca58860eba94ad |
| institution | Kabale University |
| issn | 1687-5281 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | EURASIP Journal on Image and Video Processing |
| spelling | doaj-art-ddf28052d0ed4730afca58860eba94ad2025-01-26T12:47:53ZengSpringerOpenEURASIP Journal on Image and Video Processing1687-52812025-01-012025112010.1186/s13640-024-00658-9Assessing bias and computational efficiency in vision transformers using early exitsSeth Nixon0Pietro Ruiu1Marinella Cadoni2Andrea Lagorio3Massimo Tistarelli4Department of Biomedical Sciences, University of SassariDepartment of Biomedical Sciences, University of SassariDepartment of Biomedical Sciences, University of SassariDepartment of Biomedical Sciences, University of SassariDepartment of Biomedical Sciences, University of SassariAbstract Face recognition with deep learning is generally approached as a problem of capacity. The field has seen progressively deeper, more complex models or larger, more highly variant data sets. The data sets can be problematic, as they are often scraped indiscriminately from the internet. This results in an uncertain, and often heavily unbalanced distribution of race, gender, age and other aspects of the subjects, which is then manifested in the decisions of the models trained on them. The carbon footprint of machine learning is a concern. A real push is developing to reduce the energy consumption of machine learning as we strive for a more eco-friendly society. In addition, due to many instances of misuse by law enforcement and other agencies, unbiased models for face recognition are now fundamental to the practical application of the field. We present an approach using the state of the art Vision Transformer and Early Exits for reducing compute budget without significantly affecting performance. We develop a system for face recognition and identification with a closed-set gallery and show that with a small reduction in performance, a reasonable reduction in compute cost can be obtained using our method. Second, we investigate how these early Exits interact with the bias model through a robust evaluation of matching scores on a racially balanced data set. We show that matching scores vary heavily between cohorts, and these variations are magnified at the earlier exits.https://doi.org/10.1186/s13640-024-00658-9BiometricsFace recognitionVision transformerEarly exitBias |
| spellingShingle | Seth Nixon Pietro Ruiu Marinella Cadoni Andrea Lagorio Massimo Tistarelli Assessing bias and computational efficiency in vision transformers using early exits EURASIP Journal on Image and Video Processing Biometrics Face recognition Vision transformer Early exit Bias |
| title | Assessing bias and computational efficiency in vision transformers using early exits |
| title_full | Assessing bias and computational efficiency in vision transformers using early exits |
| title_fullStr | Assessing bias and computational efficiency in vision transformers using early exits |
| title_full_unstemmed | Assessing bias and computational efficiency in vision transformers using early exits |
| title_short | Assessing bias and computational efficiency in vision transformers using early exits |
| title_sort | assessing bias and computational efficiency in vision transformers using early exits |
| topic | Biometrics Face recognition Vision transformer Early exit Bias |
| url | https://doi.org/10.1186/s13640-024-00658-9 |
| work_keys_str_mv | AT sethnixon assessingbiasandcomputationalefficiencyinvisiontransformersusingearlyexits AT pietroruiu assessingbiasandcomputationalefficiencyinvisiontransformersusingearlyexits AT marinellacadoni assessingbiasandcomputationalefficiencyinvisiontransformersusingearlyexits AT andrealagorio assessingbiasandcomputationalefficiencyinvisiontransformersusingearlyexits AT massimotistarelli assessingbiasandcomputationalefficiencyinvisiontransformersusingearlyexits |