MSSAM: A robust multi-scale adaptation of Segment Anything Model for general medical image segmentation

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

Medical image segmentation aims to identify and segment key regions such as organs, lesions, and cancers, playing a crucial role in clinical diagnosis. Although the Segment Anything Model (SAM) has achieved excellent results in image segmentation, issues like complex human organ morphology and image blurriness limit its effectiveness. Some researchers’ attempts to fine-tune SAM have yielded unsatisfactory outcomes. To address these problems, this paper proposes a new general medical segmentation model (MSSAM). Specifically, MSSAM efficiently transfers SAM to the field of medical image segmentation, while also enhancing the robustness of model training. By introducing multi-scale feature generators and decoders, MSSAM can better handle complex texture and semantic information in medical images. Through multi-scale processing, it gradually fuses low-level texture features and high-level semantic information, enabling accurate segmentation of medical images with different sizes and complex structures. To enable efficient fine-tuning of SAM on medical image datasets, the Low-Rank Adaptation (LoRA) and Adapter fine-tuning strategies are introduced in Transformer Block. Their combination significantly improves SAM’s adaptability to medical images with minimal parameters and computational overhead. Extensive experimental evaluations show that the proposed model outperforms existing SOTA segmentation models, especially demonstrating higher robustness and accuracy when processing medical images with complex structures and significant texture variations. MSSAM demonstrates consistent improvements in Dice Coefficient, Intersection-Over-Union (IoU), Accuracy, and Sensitivity across BUSI, Synapse, ACDC, and Polyp datasets. Notably, it achieves an 8.24% improvement over the second-best SOTA method on the ColonDB dataset, underscoring its effectiveness and generality in medical image segmentation.