Data

Tools

Indexes

Applications

Experiments

Open Source Science

089-sali-short-term-alignment-and-long-term-interaction-network-for-colonoscopy-video-polyp-segment

https://github.com/szu-advtech-2024/089-sali-short-term-alignment-and-long-term-interaction-network-for-colonoscopy-video-polyp-segment

Science Score: 23.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ✓
    Academic publication links
    Links to: scholar.google
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (6.9%) to scientific vocabulary
Last synced: 7 months ago · JSON representation

Repository

Basic Info
  • Host: GitHub
  • Owner: SZU-AdvTech-2024
  • Default Branch: main
  • Size: 0 Bytes
Statistics
  • Stars: 0
  • Watchers: 0
  • Forks: 0
  • Open Issues: 0
  • Releases: 0
Created about 1 year ago · Last pushed about 1 year ago
Metadata Files
Citation

https://github.com/SZU-AdvTech-2024/089-SALI-Short-term-Alignment-and-Long-term-Interaction-Network-for-Colonoscopy-Video-Polyp-Segment/blob/main/ 



SALI


Short-term Alignment and Long-term Interaction Network for Colonoscopy Video Polyp Segmentation



Qiang Hu1, *, Zhenyu Yi2, *, Ying Zhou1, Fan Huang3, Mei Liu4, Qiang Li1, Zhiwei Wang1, †



1  WNLO, HUST, 2  SES, HUST, 3  UIH, 4 HUST Tongji Medical College


(*: equal contribution, †: corresponding author)




[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/sali-short-term-alignment-and-long-term-1/video-polyp-segmentation-on-sun-seg-easy)](https://paperswithcode.com/sota/video-polyp-segmentation-on-sun-seg-easy?p=sali-short-term-alignment-and-long-term-1) 
[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/sali-short-term-alignment-and-long-term-1/video-polyp-segmentation-on-sun-seg-hard)](https://paperswithcode.com/sota/video-polyp-segmentation-on-sun-seg-hard?p=sali-short-term-alignment-and-long-term-1)
## Methods


Colonoscopy videos provide richer information in polyp segmentation for rectal cancer diagnosis.However, the endoscope's fast moving and close-up observing make the current methods suffer from large spatial incoherence and continuous low-quality frames, and thus yield limited segmentation accuracy. In this context, we focus on robust video polyp segmentation by enhancing the adjacent feature consistency and rebuilding the reliable polyp representation. To achieve this goal, we in this paper propose SALI network, a hybrid of Short-term Alignment Module (SAM) and Long-term Interaction Module (LIM).The SAM learns spatial-aligned features of adjacent frames via deformable convolution and further harmonizes them to capture more stable short-term polyp representation. In case of low-quality frames, the LIM stores the historical polyp representations as a long-term memory bank, and explores the retrospective relations to interactively rebuild more reliable polyp features for the current segmentation. Combing SAM and LIM, the SALI network of video segmentation shows a great robustness to the spatial variations and low-visual cues. SALI showcases formidable Learning Ability (`92.7/89.1` max Dice score on SUN-SEG-Seen-Easy/-Hard) and Generalization Capabilities (`82.5/82.2` max Dice score on SUN-SEG-Unseen-Easy/-Hard) in the VPS task, surpassing previous models by a large margin. ## Experimental Results ### - Performance



### - Stability and reliability of the features ### - Stability


### - Realiability


- The figure below illustrates some of the `Consecutive Low-quality Sequences` in the specific sub-test set.


## Usage ### - Preliminaries - Python 3.8+ - PyTorch 1.9+ - TorchVision corresponding to the PyTorch version - NVIDIA GPU + [CUDA](https://developer.nvidia.com/cuda-downloads) #### 1. Install dependencies for SALI. ```bash # Install other dependent packages pip install -r requirements.txt # Install cuda extensions for FA cd lib/ops_align python setup.py build develop cd ../.. ``` #### 2. Prepare the datasets for SALI. Please refer to [PNS+](https://github.com/GewelsJI/VPS/blob/main/docs/DATA_DESCRIPTION.md) to get access to the SUN-SEG dataset, and download it to path `./datasets`. The path structure should be as follows: ```none SALI datasets SUN-SEG TestEasyDataset Seen Unseen TestHardDataset Seen Unseen TrainDataset ``` #### 3. Prepare the pre-trained weights for the backbone. The pre-trained weights is available [here](https://drive.google.com/file/d/1U77oKKK_qik2C0fd7hSKiYG43UA25GgD/view?usp=sharing). ```bash mkdir pretrained cd pretrained # download the weights with the links above. ``` ### - Training ```bash python train_video.py ``` ### - Testing ```bash python test_video.py ``` ### - Well trained model: You can download our [checkpoint](https://drive.google.com/file/d/1sZvcWk2FFQo_6c6xFORLp-NjPwrptsAH/view?usp=sharing) and put it in directory `./snapshot` for a quick test. ### - Evaluating For fair comparison, we evaluate all methods through the toolbox `./eval` provided by [PNS+](https://github.com/GewelsJI/VPS/tree/main/eval). ### - Pre-computed maps: The predition maps of SALI can be downloaded via this [link](https://drive.google.com/file/d/1L1ZcSUZxTJqRPoMjUaRRFzXlXdmApSOx/view?usp=drive_link). ## Citation If you find our paper and code useful in your research, please consider giving us a star and citing SALI by the following BibTeX entry. ```bash @inproceedings{hu2024sali, title={SALI: Short-Term Alignment and Long-Term Interaction Network for Colonoscopy Video Polyp Segmentation}, author={Hu, Qiang and Yi, Zhenyu and Zhou, Ying and Peng, Fang and Liu, Mei and Li, Qiang and Wang, Zhiwei}, booktitle={International Conference on Medical Image Computing and Computer-Assisted Intervention}, pages={531--541}, year={2024}, organization={Springer} } ```

Owner

  • Name: SZU-AdvTech-2024
  • Login: SZU-AdvTech-2024
  • Kind: organization

GitHub Events

Total
  • Push event: 1
  • Create event: 3
Last Year
  • Push event: 1
  • Create event: 3