ADA-NAF: Semi-Supervised Anomaly Detection Based on the Neural Attention Forest
In this study, we present a novel model called ADA-NAF (Anomaly Detection Autoencoder with the Neural Attention Forest) for semi-supervised anomaly detection that uniquely integrates the Neural Attention Forest (NAF) architecture which has been developed to combine a random forest classifier with a...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Russian Academy of Sciences, St. Petersburg Federal Research Center
2025-01-01
|
| Series: | Информатика и автоматизация |
| Subjects: | |
| Online Access: | https://ia.spcras.ru/index.php/sp/article/view/16598 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832592315464548352 |
|---|---|
| author | Andrey Ageev Andrei Konstantinov Lev Utkin |
| author_facet | Andrey Ageev Andrei Konstantinov Lev Utkin |
| author_sort | Andrey Ageev |
| collection | DOAJ |
| description | In this study, we present a novel model called ADA-NAF (Anomaly Detection Autoencoder with the Neural Attention Forest) for semi-supervised anomaly detection that uniquely integrates the Neural Attention Forest (NAF) architecture which has been developed to combine a random forest classifier with a neural network computing attention weights to aggregate decision tree predictions. The key idea behind ADA-NAF is the incorporation of NAF into an autoencoder structure, where it implements functions of a compressor as well as a reconstructor of input vectors. Our approach introduces several technical advances. First, a proposed end-to-end training methodology over normal data minimizes the reconstruction errors while learning and optimizing neural attention weights to focus on hidden features. Second, a novel encoding mechanism leverages NAF’s hierarchical structure to capture complex data patterns. Third, an adaptive anomaly scoring framework combines the reconstruction errors with the attention-based feature importance. Through extensive experimentation across diverse datasets, ADA-NAF demonstrates superior performance compared to state-of-the-art methods. The model shows particular strength in handling high-dimensional data and capturing subtle anomalies that traditional methods often do not detect. Our results validate the ADA-NAF’s effectiveness and versatility as a robust solution for real-world anomaly detection challenges with promising applications in cybersecurity, industrial monitoring, and healthcare diagnostics. This work advances the field by introducing a novel architecture that combines the interpretability of attention mechanisms with the powerful feature learning capabilities of autoencoders. |
| format | Article |
| id | doaj-art-d5de3701deea4ac3865e00fc79baee06 |
| institution | Kabale University |
| issn | 2713-3192 2713-3206 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Russian Academy of Sciences, St. Petersburg Federal Research Center |
| record_format | Article |
| series | Информатика и автоматизация |
| spelling | doaj-art-d5de3701deea4ac3865e00fc79baee062025-01-21T11:27:24ZengRussian Academy of Sciences, St. Petersburg Federal Research CenterИнформатика и автоматизация2713-31922713-32062025-01-0124132935710.15622/ia.24.1.1216598ADA-NAF: Semi-Supervised Anomaly Detection Based on the Neural Attention ForestAndrey Ageev0Andrei Konstantinov1Lev Utkin2Peter the Great St. Petersburg Polytechnic UniversityPeter the Great St. Petersburg Polytechnic UniversityPeter the Great St. Petersburg Polytechnic UniversityIn this study, we present a novel model called ADA-NAF (Anomaly Detection Autoencoder with the Neural Attention Forest) for semi-supervised anomaly detection that uniquely integrates the Neural Attention Forest (NAF) architecture which has been developed to combine a random forest classifier with a neural network computing attention weights to aggregate decision tree predictions. The key idea behind ADA-NAF is the incorporation of NAF into an autoencoder structure, where it implements functions of a compressor as well as a reconstructor of input vectors. Our approach introduces several technical advances. First, a proposed end-to-end training methodology over normal data minimizes the reconstruction errors while learning and optimizing neural attention weights to focus on hidden features. Second, a novel encoding mechanism leverages NAF’s hierarchical structure to capture complex data patterns. Third, an adaptive anomaly scoring framework combines the reconstruction errors with the attention-based feature importance. Through extensive experimentation across diverse datasets, ADA-NAF demonstrates superior performance compared to state-of-the-art methods. The model shows particular strength in handling high-dimensional data and capturing subtle anomalies that traditional methods often do not detect. Our results validate the ADA-NAF’s effectiveness and versatility as a robust solution for real-world anomaly detection challenges with promising applications in cybersecurity, industrial monitoring, and healthcare diagnostics. This work advances the field by introducing a novel architecture that combines the interpretability of attention mechanisms with the powerful feature learning capabilities of autoencoders.https://ia.spcras.ru/index.php/sp/article/view/16598anomaly detectionrandom forestattention mechanismneural attention forest |
| spellingShingle | Andrey Ageev Andrei Konstantinov Lev Utkin ADA-NAF: Semi-Supervised Anomaly Detection Based on the Neural Attention Forest Информатика и автоматизация anomaly detection random forest attention mechanism neural attention forest |
| title | ADA-NAF: Semi-Supervised Anomaly Detection Based on the Neural Attention Forest |
| title_full | ADA-NAF: Semi-Supervised Anomaly Detection Based on the Neural Attention Forest |
| title_fullStr | ADA-NAF: Semi-Supervised Anomaly Detection Based on the Neural Attention Forest |
| title_full_unstemmed | ADA-NAF: Semi-Supervised Anomaly Detection Based on the Neural Attention Forest |
| title_short | ADA-NAF: Semi-Supervised Anomaly Detection Based on the Neural Attention Forest |
| title_sort | ada naf semi supervised anomaly detection based on the neural attention forest |
| topic | anomaly detection random forest attention mechanism neural attention forest |
| url | https://ia.spcras.ru/index.php/sp/article/view/16598 |
| work_keys_str_mv | AT andreyageev adanafsemisupervisedanomalydetectionbasedontheneuralattentionforest AT andreikonstantinov adanafsemisupervisedanomalydetectionbasedontheneuralattentionforest AT levutkin adanafsemisupervisedanomalydetectionbasedontheneuralattentionforest |