Frequency-aware sparse token vision transformer for medical image segmentation

Counts differ because each database indexes a different set of publications. We treat OpenAlex as the canonical count; Google Scholar is not shown (no API, and crawling it violates its ToS).

Accurate segmentation of organs and lesions in medical images is essential for clinical applications such as diagnosis, prognosis, and treatment planning. Vision transformer (ViT)-based models have demonstrated strong performance in medical image segmentation by modeling long-range dependencies through self-attention mechanisms, but these models still face several challenges, including sensitivity to noise, limited ability to capture fine-grained structures, and high computational overhead caused by dense token representations. To address these limitations, we propose a frequency-aware sparse token vision transformer (FS-ViT) for medical image segmentation. The proposed framework incorporates two key modules: a frequency-aware sparse self-attention (FS-SA) module and a frequency-aware MLP (FA-MLP) module. The FS-SA module first transforms spatial tokens into the frequency domain using Fourier transformation and then calculates self-attention scores to capture robust global representations while reducing sensitivity to spatial noise. In addition, a sparse token mechanism is introduced to eliminate redundant tokens, thus improving computational efficiency by lowering overhead. Additionally, the FA-MLP module is proposed to enhance feature representation by modeling fine-grained structures in the frequency domain by employing Fourier-based feature learning and a learnable query mechanism. The FS-SA and FA-MLP modules are integrated into a U-shaped encoder–decoder ViT architecture. The proposed method is evaluated on five medical image segmentation tasks, and the experimental results demonstrate that FS-ViT achieved superior performance compared with other state-of-the-art segmentation methods, highlighting its effectiveness and generalizability for robust and efficient medical image segmentation.