089-dn-detr-accelerate-detr-training-by-introducing-query-denoising

https://github.com/szu-advtech-2023/089-dn-detr-accelerate-detr-training-by-introducing-query-denoising

Repository

https://github.com/SZU-AdvTech-2023/089-DN-DETR-Accelerate-DETR-Training-by-Introducing-Query-DeNoising/blob/main/

**DN-DETR**: Accelerate DETR Training by Introducing Query DeNoising
========

By [Feng Li*](https://fengli-ust.github.io/), [Hao Zhang*](https://haozhang534.github.io/), [Shilong Liu](https://scholar.google.com/citations?hl=zh-CN&user=nkSVY3MAAAAJ), [Jian Guo](https://idea.edu.cn/en/about-team/jian_guo.html), [Lionel M.Ni](https://scholar.google.com/citations?hl=zh-CN&user=OzMYwDIAAAAJ), and [Lei Zhang](https://scholar.google.com/citations?hl=zh-CN&user=fIlGZToAAAAJ).

This repository is an official implementation of the [DN-DETR](https://arxiv.org/pdf/2203.01305.pdf). Accepted to **CVPR 2022** (score **112**, **Oral** presentation). Code is avaliable now.
[[CVPR paper link](https://openaccess.thecvf.com/content/CVPR2022/papers/Li_DN-DETR_Accelerate_DETR_Training_by_Introducing_Query_DeNoising_CVPR_2022_paper.pdf)] [[extended version paper link](https://arxiv.org/pdf/2203.01305.pdf)] [[](https://www.zhihu.com/question/517340666/answer/2381304399)]


## Introduction
 DN-DETR DETRDEtection TRansformer DETR DN-DETR  DETR  Transformer GT bounding boxesDN-DETR DETR-likeDN-DETR 


## Method
 DAB-DETR 
 ground truthGT
 Transformer  ( x , y , w , h )  Transformer  GT boxes vanilla DETR  DETR 

![DN-DETR](.github/architect.png)


![DN-DETR](.github/convergence.png)


## Model

#### 50 epoch setting

  

    

      

      
name
      
backbone
      
box AP
      
Log/Config/Checkpoint

  
  

    

      
0
      
DN-DETR-R50
      
R50
      
44.4
      
Google Drive / BaiDu 
      
Table 1
    
    

      
2
      
DN-DETR-R50-DC5
      
R50
      
46.3
      
Google Drive / BaiDu 
      
Table 1
    
    

      
5
      
DN-DAB-Deformbale-DETR
(Deformbale Encoder Only)
      
R50
      
48.6
      
Google Drive / BaiDu 
      
Table 3
    
    

      
6
      
DN-DAB-Deformable-DETR-R50-v2
      
R50
      
49.5 (48.4 in 24 epochs)
      
Google Drive / BaiDu 
      
Optimized implementation with deformable attention in both encoder and decoder. See DAB-DETR for more details.
    
  




# Installation


 ```python=3.7.3,pytorch=1.9.0,cuda=11.1```. 

pytorchtorchvisionhttps://pytorch.org/get-started/locally/.



```sh
pip install -r requirements.txt
```
CUDA
```sh
cd models/dn_dab_deformable_detr/ops
python setup.py build install
# unit test (should see all checking is True)
python test.py
cd ../../..
```

## Data

[COCO 2017](https://cocodataset.org/) 
```
COCODIR/
   train2017/
   val2017/
   annotations/
  	 instances_train2017.json
  	 instances_val2017.json
```


## Training
DN-DETR-R50DN-Deformable-DETR-R50

[DN-DETR-R50](https://drive.google.com/drive/folders/1kuwScU8PhN61qQOl5bbiPhKAYbzDHsWs?usp=sharing)

[DN-DAB-Deformable-DETR](https://drive.google.com/drive/folders/1pIllR0VfSIqX8TmQy0PFNiPdp87j-78j?usp=sharing)

[DN-DAB-Deformable-DETR_Deformable_Encoder_Only](https://drive.google.com/drive/folders/1TLIuvMw6F9lBv77gWQ3Qcn5tdfG7kqdU?usp=sharing)


```sh
# for dn_detr: 
python main.py -m dn_dab_detr \
  --output_dir logs/dn_DABDETR/R50 \
  --batch_size 1 \
  --coco_path /path/to/your/COCODIR \ # replace the args to your COCO path
  --resume /path/to/our/checkpoint \ # replace the args to your checkpoint path
  --use_dn \
  --eval

# for dn_deformable_detr
python main.py -m dn_deformable_detr \
  --output_dir logs/dab_deformable_detr/R50 \
  --batch_size 1 \
  --coco_path /path/to/your/COCODIR \ # replace the args to your COCO path
  --resume /path/to/our/checkpoint \ # replace the args to your checkpoint path
  --transformer_activation relu \
  --use_dn \
  --eval
  
# for dn_deformable_detr_deformable_encoder_only: 
python main.py -m dn_dab_deformable_detr_deformable_encoder_only 
  --output_dir logs/dab_deformable_detr/R50 \
  --batch_size 1 \
  --coco_path /path/to/your/COCODIR \ # replace the args to your COCO path
  --resume /path/to/our/checkpoint \ # replace the args to your checkpoint path
  --transformer_activation relu \
  --num_patterns 3 \  # use 3 pattern embeddings
  --use_dn  \
  --eval

# for dn_detr
python main.py -m dn_dab_detr \
  --output_dir logs/dn_DABDETR/R50 \
  --batch_size 1 \
  --epochs 50 \
  --lr_drop 40 \
  --coco_path /path/to/your/COCODIR  # replace the args to your COCO path
  --use_dn
```


### Distributed Run

 Slurm 
```sh
# for dn_detr:
python run_with_submitit.py \
  --timeout 3000 \
  --job_name DNDETR \
  --coco_path /path/to/your/COCODIR \
  -m dn_dab_detr \
  --job_dir logs/dn_DABDETR/R50_%j \
  --batch_size 2 \
  --ngpus 8 \
  --nodes 1 \
  --epochs 50 \
  --lr_drop 40 \
  --use_dn

# for dn_dab_deformable_detr: 
python run_with_submitit.py \
  --timeout 3000 \
  --job_name dn_dab_deformable_detr \
  --coco_path /path/to/your/COCODIR \
  -m dab_deformable_detr \
  --transformer_activation relu \
  --job_dir logs/dn_dab_deformable_detr/R50_%j \
  --batch_size 2 \
  --ngpus 8 \
  --nodes 1 \
  --epochs 50 \
  --lr_drop 40 \
  --use_dn

# for dn_dab_deformable_detr_deformable_encoder_only: 
python run_with_submitit.py \
  --timeout 3000 \
  --job_name dn_dab_deformable_detr_deformable_encoder_only \
  --coco_path /path/to/your/COCODIR \
  -m dn_dab_deformable_detr_deformable_encoder_only \
  --transformer_activation relu \
  --job_dir logs/dn_dab_deformable_detr/R50_%j \
  --num_patterns 3 \ 
  --batch_size 1 \
  --ngpus 8 \
  --nodes 2 \
  --epochs 50 \
  --lr_drop 40 \
  --use_dn
```

Owner

Citation (citation.txt)

@inproceedings{REPO089,
    author = "Li, Feng and Zhang, Hao and Liu, Shilong and Guo, Jian and Ni, Lionel M and Zhang, Lei",
    booktitle = "Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition",
    pages = "13619--13627",
    title = "{DN-DETR: Accelerate DETR Training by Introducing Query DeNoising}",
    year = "2022"
}

GitHub Events