051-multimodal-industrial-anomaly-detection-via-hybrid-fusion

https://github.com/szu-advtech-2024/051-multimodal-industrial-anomaly-detection-via-hybrid-fusion

Repository

https://github.com/SZU-AdvTech-2024/051-Multimodal-Industrial-Anomaly-Detection-via-Hybrid-Fusion/blob/main/

# 3D



## 1.

```bash
pip install -r requirement.txt
# install knn_cuda
pip install --upgrade https://github.com/unlimblue/KNN_CUDA/releases/download/0.2/KNN_CUDA-0.2-py3-none-any.whl
# install pointnet2_ops_lib
pip install "git+git://github.com/erikwijmans/Pointnet2_PyTorch.git#egg=pointnet2_ops&subdirectory=pointnet2_ops_lib"
```



```txt
Ubuntu 16.04 
Python 3.8.20
torch 1.9.0+cu111
torchvision 0.10.0+cu111
torchaudio 0.9.0
numpy 1.24.4
imageio 2.35.1
```

## 2.

MVTec 3D-ADhttps://www.mvtec.com/company/research/datasets/mvtec-3d-ad

dataset

## 3. 

preprocessing.py

```shell
python utils/preprocessing.py datasets/mvtec3d/
```

## 4. 

| Backbone          | Pretrain Method                                              |
| ----------------- | ------------------------------------------------------------ |
| Point Transformer | [Point-MAE](https://drive.google.com/file/d/1-wlRIz0GM8o6BuPTJz4kTt6c_z1Gh6LX/view?usp=sharing) |
| ViT-b/8           | [DINO](https://drive.google.com/file/d/17s6lwfxwG_nf1td6LXunL-LjRaX67iyK/view?usp=sharing) |
| ViT-b/8           | [Supervised ImageNet 1K](https://storage.googleapis.com/vit_models/augreg/B_8-i21k-300ep-lr_0.001-aug_medium1-wd_0.1-do_0.0-sd_0.0--imagenet2012-steps_20k-lr_0.01-res_224.npz) |
| ViT-b/8           | [Supervised ImageNet 21K](https://storage.googleapis.com/vit_models/augreg/B_8-i21k-300ep-lr_0.001-aug_medium1-wd_0.1-do_0.0-sd_0.0.npz) |
| ViT-s/8           | [DINO](https://dl.fbaipublicfiles.com/dino/dino_deitsmall8_pretrain/dino_deitsmall8_pretrain.pth) |
| UFF               | [UFF Module](https://drive.google.com/file/d/1Z2AkfPqenJEv-IdWhVdRcvVQAsJC4DxW/view?usp=sharing) |

## 5. 

* DINO+Point_MAERGBsave_feature:datasets/patch_libUFF

  ```bash
  mkdir -p datasets/patch_lib
  python3 main.py \
  --method_name DINO+Point_MAE \
  --memory_bank multiple \
  --save_feature
  ```

* UFFcheckpoint

  ```bash
  OMP_NUM_THREADS=1 python3 -m torch.distributed.launch --nproc_per_node=1 fusion_pretrain.py    \
  --accum_iter 16 \
  --epoch 5 \
  --lr 0.003 \
  --batch_size 16 \
  --data_path datasets/patch_lib \
  --output_dir checkpoints
  ```

* M3DMuse_uff

  ```bash
  python3 main.py \
  --method_name DINO+Point_MAE+Fusion \
  --use_uff \
  --memory_bank multiple \
  --fusion_module_path checkpoints/{FUSION_CHECKPOINT}.pth
  ```

* RGB,method_nameDINO  Point_MAE, memory_banksingle

Owner

Citation (citation.txt)

@article{REPO051,
    author = "Wang, Yue and Peng, Jinlong and Zhang, Jiangning and Yi, Ran and Wang, Yabiao and Wang, Chengjie",
    journal = "2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)",
    pages = "8032-8041",
    title = "{Multimodal Industrial Anomaly Detection via Hybrid Fusion}",
    year = "2023"
}

GitHub Events