Segment anything model for few-shot medical image segmentation with domain tuning
Abstract Medical image segmentation constitutes a crucial step in the analysis of medical images, possessing extensive applications and research significance within the realm of medical research and practice. Convolutional neural network achieved great success in medical image segmentation. However,...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Springer
2024-11-01
|
| Series: | Complex & Intelligent Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1007/s40747-024-01625-7 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832571168410828800 |
|---|---|
| author | Weili Shi Penglong Zhang Yuqin Li Zhengang Jiang |
| author_facet | Weili Shi Penglong Zhang Yuqin Li Zhengang Jiang |
| author_sort | Weili Shi |
| collection | DOAJ |
| description | Abstract Medical image segmentation constitutes a crucial step in the analysis of medical images, possessing extensive applications and research significance within the realm of medical research and practice. Convolutional neural network achieved great success in medical image segmentation. However, acquiring large labeled datasets remains unattainable due to the substantial expertise and time required for image labeling, as well as heightened patient privacy concerns. To solve scarce medical image data, we propose a powerful network Domain Tuning SAM for Medical images (DT-SAM). We construct an encoder utilizing a parameter-effective fine-tuning strategy and SAM. This strategy selectively updates a small fraction of the weight increments while preserving the majority of the pre-training weights in the SAM encoder, consequently reducing the required number of training samples. Meanwhile, our approach leverages only SAM encoder structure while incorporating a decoder similar to U-Net decoder structure and redesigning skip connections to concatenate encoder-extracted features, which effectively decode the features extracted by the encoder and preserve edge information. We have conducted comprehensive experiments on three publicly available medical image segmentation datasets. The combined experimental results show that our method can effectively perform few shot medical image segmentation. With just one labeled data, achieving a Dice score of 63.51%, a HD of 17.94 and an IoU score of 73.55% on Heart Task, on Prostate Task, an average Dice score of 46.01%, a HD of 10.25 and an IoU score of 65.92% were achieved, and the Dice, HD, and IoU score reaching 88.67%, 10.63, and 90.19% on BUSI. Remarkably, with few training samples, our method consistently outperforms various based on SAM and CNN. |
| format | Article |
| id | doaj-art-5cbc5278a6524af59a9e913f37cf81a4 |
| institution | Kabale University |
| issn | 2199-4536 2198-6053 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | Springer |
| record_format | Article |
| series | Complex & Intelligent Systems |
| spelling | doaj-art-5cbc5278a6524af59a9e913f37cf81a42025-02-02T12:50:05ZengSpringerComplex & Intelligent Systems2199-45362198-60532024-11-0111111710.1007/s40747-024-01625-7Segment anything model for few-shot medical image segmentation with domain tuningWeili Shi0Penglong Zhang1Yuqin Li2Zhengang Jiang3School of Computer Science and Technology, Changchun University of Science and TechnologySchool of Computer Science and Technology, Changchun University of Science and TechnologySchool of Computer Science and Technology, Changchun University of Science and TechnologySchool of Computer Science and Technology, Changchun University of Science and TechnologyAbstract Medical image segmentation constitutes a crucial step in the analysis of medical images, possessing extensive applications and research significance within the realm of medical research and practice. Convolutional neural network achieved great success in medical image segmentation. However, acquiring large labeled datasets remains unattainable due to the substantial expertise and time required for image labeling, as well as heightened patient privacy concerns. To solve scarce medical image data, we propose a powerful network Domain Tuning SAM for Medical images (DT-SAM). We construct an encoder utilizing a parameter-effective fine-tuning strategy and SAM. This strategy selectively updates a small fraction of the weight increments while preserving the majority of the pre-training weights in the SAM encoder, consequently reducing the required number of training samples. Meanwhile, our approach leverages only SAM encoder structure while incorporating a decoder similar to U-Net decoder structure and redesigning skip connections to concatenate encoder-extracted features, which effectively decode the features extracted by the encoder and preserve edge information. We have conducted comprehensive experiments on three publicly available medical image segmentation datasets. The combined experimental results show that our method can effectively perform few shot medical image segmentation. With just one labeled data, achieving a Dice score of 63.51%, a HD of 17.94 and an IoU score of 73.55% on Heart Task, on Prostate Task, an average Dice score of 46.01%, a HD of 10.25 and an IoU score of 65.92% were achieved, and the Dice, HD, and IoU score reaching 88.67%, 10.63, and 90.19% on BUSI. Remarkably, with few training samples, our method consistently outperforms various based on SAM and CNN.https://doi.org/10.1007/s40747-024-01625-7Medical image segmentationConvolutional neural networkSegment anything modelFine-tuningFew shot segmentation |
| spellingShingle | Weili Shi Penglong Zhang Yuqin Li Zhengang Jiang Segment anything model for few-shot medical image segmentation with domain tuning Complex & Intelligent Systems Medical image segmentation Convolutional neural network Segment anything model Fine-tuning Few shot segmentation |
| title | Segment anything model for few-shot medical image segmentation with domain tuning |
| title_full | Segment anything model for few-shot medical image segmentation with domain tuning |
| title_fullStr | Segment anything model for few-shot medical image segmentation with domain tuning |
| title_full_unstemmed | Segment anything model for few-shot medical image segmentation with domain tuning |
| title_short | Segment anything model for few-shot medical image segmentation with domain tuning |
| title_sort | segment anything model for few shot medical image segmentation with domain tuning |
| topic | Medical image segmentation Convolutional neural network Segment anything model Fine-tuning Few shot segmentation |
| url | https://doi.org/10.1007/s40747-024-01625-7 |
| work_keys_str_mv | AT weilishi segmentanythingmodelforfewshotmedicalimagesegmentationwithdomaintuning AT penglongzhang segmentanythingmodelforfewshotmedicalimagesegmentationwithdomaintuning AT yuqinli segmentanythingmodelforfewshotmedicalimagesegmentationwithdomaintuning AT zhengangjiang segmentanythingmodelforfewshotmedicalimagesegmentationwithdomaintuning |