The Segment Anything Model (SAM) is a large-scale model developed for general segmentation tasks in computer vision. Trained on a substantial dataset, SAM can accurately segment various objects in natural scene images. However, due to significant semantic differences between medical and natural images, directly applying SAM to medical image segmentation does not yield optimal results. Therefore, effectively utilizing such a comprehensive foundation model for medical image analysis is an emerging research topic. Despite SAM’s current suboptimal performance in medical image segmentation, it shows preliminary recognition and localization of tissues and lesions that radiologists focus on in medical images. This implies that SAM’s generated masks, features, and stability scores hold potential value for medical image diagnosis. Therefore, based on the model output of SAM, this study introduces a SAM-based Image Enhancement (SAM-IE) method for disease diagnosis. Targeting popular medical image classification models (e.g., ResNet50 and Swin Transformer), SAM-IE is proposed to enhance image inputs by combining the binary mask and contour mask generated by SAM with the original image to create attention maps, thereby improving diagnostic performance. To validate the effectiveness of SAM-IE for diagnosis, experiments were conducted on four medical image datasets for eight classification tasks. The results demonstrate the effectiveness of our proposed SAM-IE model, showcasing SAM’s potential value in medical image classification. This study provides a feasible approach for integrating SAM into disease diagnosis.

中文翻译：

---

SAM-IE：基于 SAM 的图像增强，通过分割基础模型促进医学图像诊断

Segment Anything Model (SAM) 是为计算机视觉中的一般分割任务而开发的大型模型。经过大量数据集的训练，SAM 可以准确分割自然场景图像中的各种对象。然而，由于医学图像和自然图像之间存在显着的语义差异，直接将 SAM 应用于医学图像分割并不能产生最佳结果。因此，有效利用这种综合基础模型进行医学图像分析是一个新兴的研究课题。尽管 SAM 目前在医学图像分割方面的性能欠佳，但它显示了放射科医生在医学图像中关注的组织和病变的初步识别和定位。这意味着 SAM 生成的掩模、特征和稳定性分数对于医学图像诊断具有潜在价值。因此，基于SAM的模型输出，本研究引入了一种基于SAM的图像增强（SAM-IE）方法进行疾病诊断。针对流行的医学图像分类模型（例如ResNet50和Swin Transformer），提出了SAM-IE，通过将SAM生成的二值掩模和轮廓掩模与原始图像相结合来增强图像输入以创建注意力图，从而提高诊断性能。为了验证 SAM-IE 诊断的有效性，在四个医学图像数据集上进行了八个分类任务的实验。结果证明了我们提出的 SAM-IE 模型的有效性，展示了 SAM 在医学图像分类中的潜在价值。本研究为将 SAM 整合到疾病诊断中提供了一种可行的方法。