MedSAM2

**Segment Anything in 3D Medical Images and Videos**

Welcome to join our mailing list to get updates. We’re also actively looking to collaborate on annotating new large-scale 3D datasets. If you have unlabeled medical images or videos and want to share them with the community, let’s connect!

Updates

• 20250705: Release Efficient MedSAM2 baseline for FLARE 2025 Pan-cancer segmentation challenge RECIST-to-3D
• 20250423: Release lung lesion segmentation dataset LUNA25-MedSAM2 for LUNA25

Installation

• Create a virtual environment: conda create -n medsam2 python=3.12 -y and conda activate medsam2
• Install PyTorch: pip install torch==2.5.1 torchvision==0.20.1 --index-url https://download.pytorch.org/whl/cu124 (Linux CUDA 12.4)
• Download code git clone https://github.com/bowang-lab/MedSAM2.git && cd MedSAM2 and run pip install -e ".[dev]"
• Download checkpoints: bash download.sh
• Optional: Please install the following dependencies for gradio

bash sudo apt-get update sudo apt-get install ffmpeg pip install gradio==3.38.0 pip install numpy==1.26.3 pip install ffmpeg-python pip install moviepy

Download annotated datasets

• CT_DeepLesion-MedSAM2

• LLD-MMRI-MedSAM2

• RVENet-MedSAM2

Note: Please also cite the raw DeepLesion, LLD-MMRI and RVENET papers when using these datasets.

Inference

3D medical image segmentation

• Colab: MedSAM2inferenceCTLesionDemo.ipynb

• CMD

bash python medsam2_infer_3D_CT.py -i CT_DeepLesion/images -o CT_DeepLesion/segmentation

Medical video segmentation

• Colab: MedSAM2InferenceVideo_Demo.ipynb

• CMD

bash python medsam2_infer_video.py -i input_video_path -m input_mask_path -o output_video_path

Gradio demo

bash python app.py

Training MedSAM2

Use FLARE25 pan-cancer CT dataset as an example. - Download sam2.1hieratiny.pt to checkpoints - Add dataset information in sam2/configs/sam2.1_hiera_tiny512_FLARE_RECIST.yaml: data -> train -> datasets - Set train_video_batch_size based on the GPU memory

bash sh single_node_train_medsam2.sh

• multi-node training

bash sbatch multi_node_train.sh

• inference with RECIST marker (simulate a box prompt on middle slice)

bash python medsam2_infer_CT_lesion_npz_recist.py

Training Efficient MedSAM2

• Train Efficient MedSAM2 on FLARE25 pan-cancer CT dataset for CPU-based inference

bash sh single_node_train_eff_medsam2_FLARE25.sh

• Inference with RECIST marker on the FLARE25 pan-cancer validation dataset.

python npz = np.load('path to/CT_Lesion_FLARE23Ts_0057.npz', allow_pickle=True) print(npz.keys()) imgs = npz['imgs'] # (D, W, H), [0, 255] recist = npz['recist'] # (D, W, H), binary RECIST marker on tumor middle slice {0, 1} gts = npz['gts'] # (D, W, H), 3D tumor ground truth mask. It will be not available in the testing set

simulate a box prompt on middle slice

bash python eff_medsam2_infer_CT_lesion_npz_recist.py

Acknowledgements

• We highly appreciate all the challenge organizers and dataset owners for providing the public datasets to the community.
• We thank Meta AI for making the source code of SAM2 and EfficientTAM publicly available. Please also cite these papers when using MedSAM2.

Bibtex

bash @article{MedSAM2, title={MedSAM2: Segment Anything in 3D Medical Images and Videos}, author={Ma, Jun and Yang, Zongxin and Kim, Sumin and Chen, Bihui and Baharoon, Mohammed and Fallahpour, Adibvafa and Asakereh, Reza and Lyu, Hongwei and Wang, Bo}, journal={arXiv preprint arXiv:2504.03600}, year={2025} } Please also cite SAM2 @inproceedings{SAM2, title={{SAM} 2: Segment Anything in Images and Videos}, author={Nikhila Ravi and Valentin Gabeur and Yuan-Ting Hu and Ronghang Hu and Chaitanya Ryali and Tengyu Ma and Haitham Khedr and Roman R{\"a}dle and Chloe Rolland and Laura Gustafson and Eric Mintun and Junting Pan and Kalyan Vasudev Alwala and Nicolas Carion and Chao-Yuan Wu and Ross Girshick and Piotr Dollar and Christoph Feichtenhofer}, booktitle={International Conference on Learning Representations}, year={2025} }

and EfficientTAM

@article{xiong2024efficienttam, title={Efficient Track Anything}, author={Yunyang Xiong, Chong Zhou, Xiaoyu Xiang, Lemeng Wu, Chenchen Zhu, Zechun Liu, Saksham Suri, Balakrishnan Varadarajan, Ramya Akula, Forrest Iandola, Raghuraman Krishnamoorthi, Bilge Soran, Vikas Chandra}, journal={preprint arXiv:2411.18933}, year={2024} }