IAR 2.0: An Algorithm for Refining Inconsistent Annotations for Time-Series Data Using Discriminative Classifiers
The performance of discriminative machine-learning classifiers, such as neural networks, is limited by training label inconsistencies. Even expert-based annotations can suffer from label inconsistencies, especially in the case of ambiguous phenomena-to-annotate. To address this, we propose a novel a...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10854471/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832575610519552000 |
|---|---|
| author | Einari Vaaras Manu Airaksinen Okko Rasanen |
| author_facet | Einari Vaaras Manu Airaksinen Okko Rasanen |
| author_sort | Einari Vaaras |
| collection | DOAJ |
| description | The performance of discriminative machine-learning classifiers, such as neural networks, is limited by training label inconsistencies. Even expert-based annotations can suffer from label inconsistencies, especially in the case of ambiguous phenomena-to-annotate. To address this, we propose a novel algorithm, iterative annotation refinement (IAR) 2.0, for refining inconsistent annotations for time-series data. IAR 2.0 uses a procedure that utilizes discriminative classifiers to iteratively combine original annotations with increasingly accurate posterior estimates of classes present in the data. Unlike most existing label refinement approaches, IAR 2.0 offers a simpler yet effective solution for resolving ambiguities in training labels, working with real label noise on time-series data instead of synthetic label noise on image data. We demonstrate the effectiveness of our algorithm through five distinct classification tasks on two highly distinct data modalities. As a result, we show that the labels produced by IAR 2.0 systematically improve classifier performance compared to using the original labels or a previous state-of-the-art method for label refinement. We also conduct a set of controlled simulations to systematically investigate when IAR 2.0 fails to improve on the original training labels. The simulation results demonstrate that IAR 2.0 improves performance in nearly all tested conditions. We also find that the decrease in performance when IAR 2.0 fails is small compared to the average performance gain when IAR 2.0 succeeds, encouraging the use of IAR 2.0 even when the nature of data is unknown. The code is freely available at <uri>https://github.com/SPEECHCOG/IAR_2</uri>. |
| format | Article |
| id | doaj-art-b26fa0c3b0724744a3613e21ee4b3a21 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-b26fa0c3b0724744a3613e21ee4b3a212025-01-31T23:05:13ZengIEEEIEEE Access2169-35362025-01-0113199791999510.1109/ACCESS.2025.353463710854471IAR 2.0: An Algorithm for Refining Inconsistent Annotations for Time-Series Data Using Discriminative ClassifiersEinari Vaaras0https://orcid.org/0000-0002-8714-6090Manu Airaksinen1Okko Rasanen2https://orcid.org/0000-0002-0537-0946Signal Processing Research Centre, Tampere University, Tampere, FinlandBABA Center, University of Helsinki, Helsinki, FinlandSignal Processing Research Centre, Tampere University, Tampere, FinlandThe performance of discriminative machine-learning classifiers, such as neural networks, is limited by training label inconsistencies. Even expert-based annotations can suffer from label inconsistencies, especially in the case of ambiguous phenomena-to-annotate. To address this, we propose a novel algorithm, iterative annotation refinement (IAR) 2.0, for refining inconsistent annotations for time-series data. IAR 2.0 uses a procedure that utilizes discriminative classifiers to iteratively combine original annotations with increasingly accurate posterior estimates of classes present in the data. Unlike most existing label refinement approaches, IAR 2.0 offers a simpler yet effective solution for resolving ambiguities in training labels, working with real label noise on time-series data instead of synthetic label noise on image data. We demonstrate the effectiveness of our algorithm through five distinct classification tasks on two highly distinct data modalities. As a result, we show that the labels produced by IAR 2.0 systematically improve classifier performance compared to using the original labels or a previous state-of-the-art method for label refinement. We also conduct a set of controlled simulations to systematically investigate when IAR 2.0 fails to improve on the original training labels. The simulation results demonstrate that IAR 2.0 improves performance in nearly all tested conditions. We also find that the decrease in performance when IAR 2.0 fails is small compared to the average performance gain when IAR 2.0 succeeds, encouraging the use of IAR 2.0 even when the nature of data is unknown. The code is freely available at <uri>https://github.com/SPEECHCOG/IAR_2</uri>.https://ieeexplore.ieee.org/document/10854471/Annotation refinementdaylong audio recordingsdiscriminative classifiershuman activity recognitioninconsistent labelslabel refinement |
| spellingShingle | Einari Vaaras Manu Airaksinen Okko Rasanen IAR 2.0: An Algorithm for Refining Inconsistent Annotations for Time-Series Data Using Discriminative Classifiers IEEE Access Annotation refinement daylong audio recordings discriminative classifiers human activity recognition inconsistent labels label refinement |
| title | IAR 2.0: An Algorithm for Refining Inconsistent Annotations for Time-Series Data Using Discriminative Classifiers |
| title_full | IAR 2.0: An Algorithm for Refining Inconsistent Annotations for Time-Series Data Using Discriminative Classifiers |
| title_fullStr | IAR 2.0: An Algorithm for Refining Inconsistent Annotations for Time-Series Data Using Discriminative Classifiers |
| title_full_unstemmed | IAR 2.0: An Algorithm for Refining Inconsistent Annotations for Time-Series Data Using Discriminative Classifiers |
| title_short | IAR 2.0: An Algorithm for Refining Inconsistent Annotations for Time-Series Data Using Discriminative Classifiers |
| title_sort | iar 2 0 an algorithm for refining inconsistent annotations for time series data using discriminative classifiers |
| topic | Annotation refinement daylong audio recordings discriminative classifiers human activity recognition inconsistent labels label refinement |
| url | https://ieeexplore.ieee.org/document/10854471/ |
| work_keys_str_mv | AT einarivaaras iar20analgorithmforrefininginconsistentannotationsfortimeseriesdatausingdiscriminativeclassifiers AT manuairaksinen iar20analgorithmforrefininginconsistentannotationsfortimeseriesdatausingdiscriminativeclassifiers AT okkorasanen iar20analgorithmforrefininginconsistentannotationsfortimeseriesdatausingdiscriminativeclassifiers |