Testing convolutional neural network based deep learning systems: a statistical metamorphic approach
Machine learning technology spans many areas and today plays a significant role in addressing a wide range of problems in critical domains, i.e., healthcare, autonomous driving, finance, manufacturing, cybersecurity, etc. Metamorphic testing (MT) is considered a simple but very powerful approach in...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
PeerJ Inc.
2025-01-01
|
| Series: | PeerJ Computer Science |
| Subjects: | |
| Online Access: | https://peerj.com/articles/cs-2658.pdf |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832574408330313728 |
|---|---|
| author | Faqeer ur Rehman Clemente Izurieta |
| author_facet | Faqeer ur Rehman Clemente Izurieta |
| author_sort | Faqeer ur Rehman |
| collection | DOAJ |
| description | Machine learning technology spans many areas and today plays a significant role in addressing a wide range of problems in critical domains, i.e., healthcare, autonomous driving, finance, manufacturing, cybersecurity, etc. Metamorphic testing (MT) is considered a simple but very powerful approach in testing such computationally complex systems for which either an oracle is not available or is available but difficult to apply. Conventional metamorphic testing techniques have certain limitations in verifying deep learning-based models (i.e., convolutional neural networks (CNNs)) that have a stochastic nature (because of randomly initializing the network weights) in their training. In this article, we attempt to address this problem by using a statistical metamorphic testing (SMT) technique that does not require software testers to worry about fixing the random seeds (to get deterministic results) to verify the metamorphic relations (MRs). We propose seven MRs combined with different statistical methods to statistically verify whether the program under test adheres to the relation(s) specified in the MR(s). We further use mutation testing techniques to show the usefulness of the proposed approach in the healthcare space and test two CNN-based deep learning models (used for pneumonia detection among patients). The empirical results show that our proposed approach uncovers 85.71% of the implementation faults in the classifiers under test (CUT). Furthermore, we also propose an MRs minimization algorithm for the CUT, thus saving computational costs and organizational testing resources. |
| format | Article |
| id | doaj-art-e42b742d9d584fc686d604eb56e86301 |
| institution | Kabale University |
| issn | 2376-5992 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | PeerJ Inc. |
| record_format | Article |
| series | PeerJ Computer Science |
| spelling | doaj-art-e42b742d9d584fc686d604eb56e863012025-02-01T15:05:09ZengPeerJ Inc.PeerJ Computer Science2376-59922025-01-0111e265810.7717/peerj-cs.2658Testing convolutional neural network based deep learning systems: a statistical metamorphic approachFaqeer ur Rehman0Clemente Izurieta1Gianforte School of Computing, Montana State University, Bozeman, Montana, United StatesGianforte School of Computing, Montana State University, Bozeman, Montana, United StatesMachine learning technology spans many areas and today plays a significant role in addressing a wide range of problems in critical domains, i.e., healthcare, autonomous driving, finance, manufacturing, cybersecurity, etc. Metamorphic testing (MT) is considered a simple but very powerful approach in testing such computationally complex systems for which either an oracle is not available or is available but difficult to apply. Conventional metamorphic testing techniques have certain limitations in verifying deep learning-based models (i.e., convolutional neural networks (CNNs)) that have a stochastic nature (because of randomly initializing the network weights) in their training. In this article, we attempt to address this problem by using a statistical metamorphic testing (SMT) technique that does not require software testers to worry about fixing the random seeds (to get deterministic results) to verify the metamorphic relations (MRs). We propose seven MRs combined with different statistical methods to statistically verify whether the program under test adheres to the relation(s) specified in the MR(s). We further use mutation testing techniques to show the usefulness of the proposed approach in the healthcare space and test two CNN-based deep learning models (used for pneumonia detection among patients). The empirical results show that our proposed approach uncovers 85.71% of the implementation faults in the classifiers under test (CUT). Furthermore, we also propose an MRs minimization algorithm for the CUT, thus saving computational costs and organizational testing resources.https://peerj.com/articles/cs-2658.pdfMetamorphic relationsMetamorphic testingTesting convolutional neural networks (CNNs)Testing deep learning systemsTesting pneumonia detection modelsMetamorphic relations prioritization |
| spellingShingle | Faqeer ur Rehman Clemente Izurieta Testing convolutional neural network based deep learning systems: a statistical metamorphic approach PeerJ Computer Science Metamorphic relations Metamorphic testing Testing convolutional neural networks (CNNs) Testing deep learning systems Testing pneumonia detection models Metamorphic relations prioritization |
| title | Testing convolutional neural network based deep learning systems: a statistical metamorphic approach |
| title_full | Testing convolutional neural network based deep learning systems: a statistical metamorphic approach |
| title_fullStr | Testing convolutional neural network based deep learning systems: a statistical metamorphic approach |
| title_full_unstemmed | Testing convolutional neural network based deep learning systems: a statistical metamorphic approach |
| title_short | Testing convolutional neural network based deep learning systems: a statistical metamorphic approach |
| title_sort | testing convolutional neural network based deep learning systems a statistical metamorphic approach |
| topic | Metamorphic relations Metamorphic testing Testing convolutional neural networks (CNNs) Testing deep learning systems Testing pneumonia detection models Metamorphic relations prioritization |
| url | https://peerj.com/articles/cs-2658.pdf |
| work_keys_str_mv | AT faqeerurrehman testingconvolutionalneuralnetworkbaseddeeplearningsystemsastatisticalmetamorphicapproach AT clementeizurieta testingconvolutionalneuralnetworkbaseddeeplearningsystemsastatisticalmetamorphicapproach |