A recurrent YOLOv8-based framework for event-based object detection
Object detection plays a crucial role in various cutting-edge applications, such as autonomous vehicles and advanced robotics systems, primarily relying on conventional frame-based RGB sensors. However, these sensors face challenges such as motion blur and poor performance under extreme lighting con...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2025-01-01
|
| Series: | Frontiers in Neuroscience |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fnins.2024.1477979/full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832591660298534912 |
|---|---|
| author | Diego A. Silva Kamilya Smagulova Ahmed Elsheikh Mohammed E. Fouda Ahmed M. Eltawil |
| author_facet | Diego A. Silva Kamilya Smagulova Ahmed Elsheikh Mohammed E. Fouda Ahmed M. Eltawil |
| author_sort | Diego A. Silva |
| collection | DOAJ |
| description | Object detection plays a crucial role in various cutting-edge applications, such as autonomous vehicles and advanced robotics systems, primarily relying on conventional frame-based RGB sensors. However, these sensors face challenges such as motion blur and poor performance under extreme lighting conditions. Novel event-based cameras, inspired by biological vision systems, offer a promising solution with superior performance in fast-motion and challenging lighting environments while consuming less power. This work explores the integration of event-based cameras with advanced object detection frameworks, introducing Recurrent YOLOv8 (ReYOLOV8), a refined object detection framework that enhances a leading frame-based YOLO detection system with spatiotemporal modeling capabilities by adding recurrency. ReYOLOv8 incorporates a low-latency, memory-efficient method for encoding event data called Volume of Ternary Event Images (VTEI) and introduces a novel data augmentation technique based on Random Polarity Suppression (RPS) optimized for event-based sensors and tailored to leverage the unique attributes of event data. The framework was evaluated using two comprehensive event-based datasets Prophesee's Generation 1 (GEN1) and Person Detection for Robotics (PEDRo). On the GEN1 dataset, ReYOLOv8 achieved mAP improvements of 5%, 2.8%, and 2.5% across nano, small, and medium scales, respectively, while reducing trainable parameters by 4.43% on average and maintaining real-time processing speeds between 9.2 ms and 15.5 ms. For the PEDRo dataset, ReYOLOv8 demonstrated mAP improvements ranging from 9% to 18%, with models reduced in size by factors of 14.5 × and 3.8 × and an average speed improvement of 1.67 × . The results demonstrate the significant potential of bio-inspired event-based vision sensors when combined with advanced object detection frameworks. In particular, the ReYOLOv8 system effectively bridges the gap between biological principles of vision and artificial intelligence, enabling robust and efficient visual processing in dynamic and complex environments. The codes are available on GitHub at the following link https://github.com/silvada95/ReYOLOv8. |
| format | Article |
| id | doaj-art-bbc0e741dc544d2cbfd0daec430c095a |
| institution | Kabale University |
| issn | 1662-453X |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neuroscience |
| spelling | doaj-art-bbc0e741dc544d2cbfd0daec430c095a2025-01-22T07:15:09ZengFrontiers Media S.A.Frontiers in Neuroscience1662-453X2025-01-011810.3389/fnins.2024.14779791477979A recurrent YOLOv8-based framework for event-based object detectionDiego A. Silva0Kamilya Smagulova1Ahmed Elsheikh2Mohammed E. Fouda3Ahmed M. Eltawil4Communication and Computing Systems Lab, Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology, Thuwal, Saudi ArabiaCommunication and Computing Systems Lab, Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology, Thuwal, Saudi ArabiaMathematics and Engineering Physics Department, Faculty of Engineering, Cairo University, Giza, EgyptCompumacy for Artificial Intelligence Solutions, Cairo, EgyptCommunication and Computing Systems Lab, Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology, Thuwal, Saudi ArabiaObject detection plays a crucial role in various cutting-edge applications, such as autonomous vehicles and advanced robotics systems, primarily relying on conventional frame-based RGB sensors. However, these sensors face challenges such as motion blur and poor performance under extreme lighting conditions. Novel event-based cameras, inspired by biological vision systems, offer a promising solution with superior performance in fast-motion and challenging lighting environments while consuming less power. This work explores the integration of event-based cameras with advanced object detection frameworks, introducing Recurrent YOLOv8 (ReYOLOV8), a refined object detection framework that enhances a leading frame-based YOLO detection system with spatiotemporal modeling capabilities by adding recurrency. ReYOLOv8 incorporates a low-latency, memory-efficient method for encoding event data called Volume of Ternary Event Images (VTEI) and introduces a novel data augmentation technique based on Random Polarity Suppression (RPS) optimized for event-based sensors and tailored to leverage the unique attributes of event data. The framework was evaluated using two comprehensive event-based datasets Prophesee's Generation 1 (GEN1) and Person Detection for Robotics (PEDRo). On the GEN1 dataset, ReYOLOv8 achieved mAP improvements of 5%, 2.8%, and 2.5% across nano, small, and medium scales, respectively, while reducing trainable parameters by 4.43% on average and maintaining real-time processing speeds between 9.2 ms and 15.5 ms. For the PEDRo dataset, ReYOLOv8 demonstrated mAP improvements ranging from 9% to 18%, with models reduced in size by factors of 14.5 × and 3.8 × and an average speed improvement of 1.67 × . The results demonstrate the significant potential of bio-inspired event-based vision sensors when combined with advanced object detection frameworks. In particular, the ReYOLOv8 system effectively bridges the gap between biological principles of vision and artificial intelligence, enabling robust and efficient visual processing in dynamic and complex environments. The codes are available on GitHub at the following link https://github.com/silvada95/ReYOLOv8.https://www.frontiersin.org/articles/10.3389/fnins.2024.1477979/fullobject detectionYOLOevent-based camerasdata augmentationautonomous driving |
| spellingShingle | Diego A. Silva Kamilya Smagulova Ahmed Elsheikh Mohammed E. Fouda Ahmed M. Eltawil A recurrent YOLOv8-based framework for event-based object detection Frontiers in Neuroscience object detection YOLO event-based cameras data augmentation autonomous driving |
| title | A recurrent YOLOv8-based framework for event-based object detection |
| title_full | A recurrent YOLOv8-based framework for event-based object detection |
| title_fullStr | A recurrent YOLOv8-based framework for event-based object detection |
| title_full_unstemmed | A recurrent YOLOv8-based framework for event-based object detection |
| title_short | A recurrent YOLOv8-based framework for event-based object detection |
| title_sort | recurrent yolov8 based framework for event based object detection |
| topic | object detection YOLO event-based cameras data augmentation autonomous driving |
| url | https://www.frontiersin.org/articles/10.3389/fnins.2024.1477979/full |
| work_keys_str_mv | AT diegoasilva arecurrentyolov8basedframeworkforeventbasedobjectdetection AT kamilyasmagulova arecurrentyolov8basedframeworkforeventbasedobjectdetection AT ahmedelsheikh arecurrentyolov8basedframeworkforeventbasedobjectdetection AT mohammedefouda arecurrentyolov8basedframeworkforeventbasedobjectdetection AT ahmedmeltawil arecurrentyolov8basedframeworkforeventbasedobjectdetection AT diegoasilva recurrentyolov8basedframeworkforeventbasedobjectdetection AT kamilyasmagulova recurrentyolov8basedframeworkforeventbasedobjectdetection AT ahmedelsheikh recurrentyolov8basedframeworkforeventbasedobjectdetection AT mohammedefouda recurrentyolov8basedframeworkforeventbasedobjectdetection AT ahmedmeltawil recurrentyolov8basedframeworkforeventbasedobjectdetection |