Non-convex optimization with using positive-negative moment estimation and its application for skin cancer recognition with a neural network
The main problem of using standard optimization methods is the need to change all parameters in same-size steps, regardless of the behavior of the gradient. A more efficient way to optimize a neural network is to set adaptive step sizes for each parameter. Standard methods are based on the square ro...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Samara National Research University
2024-04-01
|
| Series: | Компьютерная оптика |
| Subjects: | |
| Online Access: | https://www.computeroptics.ru/eng/KO/Annot/KO48-2/480213e.html |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832540878216888320 |
|---|---|
| author | P.A. Lyakhov U.A. Lyakhova R.I. Abdulkadirov |
| author_facet | P.A. Lyakhov U.A. Lyakhova R.I. Abdulkadirov |
| author_sort | P.A. Lyakhov |
| collection | DOAJ |
| description | The main problem of using standard optimization methods is the need to change all parameters in same-size steps, regardless of the behavior of the gradient. A more efficient way to optimize a neural network is to set adaptive step sizes for each parameter. Standard methods are based on the square roots of exponential estimates of the moments of the squares of past gradients and do not use the local variation in gradients. The paper presents methods of adaptive non-convex and belief-based optimization with a positive-negative estimate of the moments with the corresponding theoretical guarantees of convergence. These approaches allow the loss function to more accurately converge in the neighborhood of the global minimum in a smaller number of iterations. The utilization of transformed positive-negative moment estimates and an additional parameter that controls the step size allows one to avoid local extremes for achieving higher performance, compared to similar methods. The introduction of the developed algorithms into the learning process of various architectures of multimodal neural network systems for analyzing heterogeneous data has made it possible to increase the accuracy of recognizing pigmented skin lesions by 2.33 – 5.69 percentage points, compared to the original optimization methods. Multimodal neural network systems for analyzing heterogeneous dermatological data, using the proposed optimization algorithms, can be applied as a tool for auxiliary medical diagnostics, which will reduce the consumption of financial and labor resources involved in the medical industry, as well as increase the chance of early detection of pigmentary oncopathologies. |
| format | Article |
| id | doaj-art-0f20c84063604b738dd27b162e56fb84 |
| institution | Kabale University |
| issn | 0134-2452 2412-6179 |
| language | English |
| publishDate | 2024-04-01 |
| publisher | Samara National Research University |
| record_format | Article |
| series | Компьютерная оптика |
| spelling | doaj-art-0f20c84063604b738dd27b162e56fb842025-02-04T12:48:30ZengSamara National Research UniversityКомпьютерная оптика0134-24522412-61792024-04-0148226027110.18287/2412-6179-CO-1308Non-convex optimization with using positive-negative moment estimation and its application for skin cancer recognition with a neural networkP.A. Lyakhov0U.A. Lyakhova1R.I. Abdulkadirov2North-Caucasus Federal University; North-Caucasus Center for Mathematical ResearchNorth-Caucasus Federal University; North-Caucasus Center for Mathematical ResearchNorth-Caucasus Center for Mathematical ResearchThe main problem of using standard optimization methods is the need to change all parameters in same-size steps, regardless of the behavior of the gradient. A more efficient way to optimize a neural network is to set adaptive step sizes for each parameter. Standard methods are based on the square roots of exponential estimates of the moments of the squares of past gradients and do not use the local variation in gradients. The paper presents methods of adaptive non-convex and belief-based optimization with a positive-negative estimate of the moments with the corresponding theoretical guarantees of convergence. These approaches allow the loss function to more accurately converge in the neighborhood of the global minimum in a smaller number of iterations. The utilization of transformed positive-negative moment estimates and an additional parameter that controls the step size allows one to avoid local extremes for achieving higher performance, compared to similar methods. The introduction of the developed algorithms into the learning process of various architectures of multimodal neural network systems for analyzing heterogeneous data has made it possible to increase the accuracy of recognizing pigmented skin lesions by 2.33 – 5.69 percentage points, compared to the original optimization methods. Multimodal neural network systems for analyzing heterogeneous dermatological data, using the proposed optimization algorithms, can be applied as a tool for auxiliary medical diagnostics, which will reduce the consumption of financial and labor resources involved in the medical industry, as well as increase the chance of early detection of pigmentary oncopathologies.https://www.computeroptics.ru/eng/KO/Annot/KO48-2/480213e.htmloptimizationnatural gradient descentartificial intelligencemultimodal neural networksheterogeneous dataskin cancermelanoma |
| spellingShingle | P.A. Lyakhov U.A. Lyakhova R.I. Abdulkadirov Non-convex optimization with using positive-negative moment estimation and its application for skin cancer recognition with a neural network Компьютерная оптика optimization natural gradient descent artificial intelligence multimodal neural networks heterogeneous data skin cancer melanoma |
| title | Non-convex optimization with using positive-negative moment estimation and its application for skin cancer recognition with a neural network |
| title_full | Non-convex optimization with using positive-negative moment estimation and its application for skin cancer recognition with a neural network |
| title_fullStr | Non-convex optimization with using positive-negative moment estimation and its application for skin cancer recognition with a neural network |
| title_full_unstemmed | Non-convex optimization with using positive-negative moment estimation and its application for skin cancer recognition with a neural network |
| title_short | Non-convex optimization with using positive-negative moment estimation and its application for skin cancer recognition with a neural network |
| title_sort | non convex optimization with using positive negative moment estimation and its application for skin cancer recognition with a neural network |
| topic | optimization natural gradient descent artificial intelligence multimodal neural networks heterogeneous data skin cancer melanoma |
| url | https://www.computeroptics.ru/eng/KO/Annot/KO48-2/480213e.html |
| work_keys_str_mv | AT palyakhov nonconvexoptimizationwithusingpositivenegativemomentestimationanditsapplicationforskincancerrecognitionwithaneuralnetwork AT ualyakhova nonconvexoptimizationwithusingpositivenegativemomentestimationanditsapplicationforskincancerrecognitionwithaneuralnetwork AT riabdulkadirov nonconvexoptimizationwithusingpositivenegativemomentestimationanditsapplicationforskincancerrecognitionwithaneuralnetwork |