377-multi-class-token-transformer-for-weakly-supervised-semantic-segmentation

https://github.com/szu-advtech-2023/377-multi-class-token-transformer-for-weakly-supervised-semantic-segmentation

Repository

https://github.com/SZU-AdvTech-2023/377-Multi-Class-Token-Transformer-for-Weakly-Supervised-Semantic-Segmentation/blob/main/

# MCTformer
The pytorch code for our CVPR2022 paper [Multi-class Token Transformer for Weakly Supervised Semantic Segmentation](https://arxiv.org/abs/2203.02891).

[[Paper]](https://arxiv.org/abs/2203.02891) [[Project Page]](https://xulianuwa.github.io/MCTformer-project-page/)


  


Fig.1 - Overview of MCTformer




## Prerequisite
- Ubuntu 18.04, with Python 3.6 and the following python dependencies.
```
pip install -r prerequisite.txt
```
- Download [the PASCAL VOC 2012 development kit](http://host.robots.ox.ac.uk/pascal/VOC/voc2012).

## Usage
Step 1: Run the run.sh script for training MCTformer, visualizing and evaluating the generated class-specific localization maps. 
```
bash runs/pascal_v2_train.sh
bash runs/pascal_v2_generate.sh
bash runs/pascal_v2_evaluate_generate.sh
```
### PASCAL VOC 2012 dataset

| Model        | Backbone   | Google drive |
|--------------|------------|--------------|
| MCTformer-V1 | DeiT-small | [Weights](https://drive.google.com/file/d/1jLnSbR2DDtjli5EwRYSDi3Xa6xxFIAi0/view?usp=sharing)  |
| MCTformer-V2 | DeiT-small | [Weights](https://drive.google.com/file/d/1w5LDoS_CHtDRXgFSqFtPvIiCajk4ZtMB/view?usp=sharing)  |

Step 2: Run the run_psa.sh script for using [PSA](https://github.com/jiwoon-ahn/psa) to post-process the seeds (i.e., class-specific localization maps) to generate pseudo ground-truth segmentation masks. To train PSA, the pre-trained classification [weights](https://drive.google.com/file/d/1xESB7017zlZHqxEWuh1Rb89UhjTGIKOA/view?usp=sharing) were used for initialization.
```
bash runs/pascal_v2_psa_train.sh
bash runs/pascal_v2_psa_infer.sh
bash runs/pascal_v2_evaluate.sh
```


Step 3: For the segmentation part, run the run_seg.sh script for training and testing the segmentation model. When training on VOC, the model was initialized with the pre-trained classification [weights](https://drive.google.com/file/d/1xESB7017zlZHqxEWuh1Rb89UhjTGIKOA/view?usp=sharing) on VOC.
```
bash runs/pascal_v2_seg_train.sh
bash runs/pascal_v2_seg_infer.sh
```

Owner

Citation (citation.txt)

@article{REPO377,
    author = "Xu, Lian and Ouyang, Wanli and Bennamoun and Boussaid, Farid and Xu, Dan",
    journal = "2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)",
    pages = "4300-4309",
    title = "{Multi-class Token Transformer for Weakly Supervised Semantic Segmentation}",
    year = "2022"
}

GitHub Events