Video-Swin-Transformer .

TSN (Temporal Segment Networks)

GitHub - SwinTransformer/Video-Swin-Transformer: This is an official implementation for "Video Swin Transformers

4. DATA SET
5. Model
6. tester()
7. recognizor()

| | (top1) | (top5) | (mean1) | | | |----------------------|--------------|--------------|--------------------|--------|--------| | bestmodel0522 | 0.2061 | 0.3876 | 0.29685 | 3.6529 | 353 MB | | bestmodel0527 | 0.2304 | 0.4683 | 0.34935 | 3.4279 | 353 MB | | bestmodel0529 | 0.2056 | 0.4503 | 0.0364 | 3.4289 | 353 MB | | bestmodel0531 | 0.1857 | 0.4206 | 0.0333 | 0.3735 | 320 MB |

mmaction2

Installation MMAction2 1.2.0 documentation

torch+torchvision

```bash

torch+torchvision

pip install torch==1.8.0+cu111 torchvision==0.9.0+cu111 -f https://download.pytorch.org/whl/torch_stable.html

mmcv

pip install mmcv-full==1.4.0 -f https://download.openmmlab.com/mmcv/dist/cu111/torch1.8.0/index.html

pip install opencv-python pip install timm pip install scipy pip install einops

pip install numpy==1.19.0 ```

Docker

```bash
ARG PYTORCH="1.6.0"
ARG CUDA="10.1"
ARG CUDNN="7"
```

• Important:Make sure you've installed thenvidia-container-toolkit.

• docker

  ```bash

  build an image with PyTorch 1.6.0, CUDA 10.1, CUDNN 7.

  docker build -f ./docker/Dockerfile --rm -t mmaction2 .

  docker run --gpus all --shm-size=8g -it -v {DATA_DIR}:/mmaction2/data mmaction2

  docker run --gpus all --shm-size=8g -it -v G:/videodatasets/downloaddatas:/mmaction2/data mmaction2

  pip install mmcv==2.1.0 pip install -r requirements/build.txt python setup.py develop

  apt-get update apt-get install wget ```

DATA SET

• download

  AI-Hub

  bash export AIHUB_ID='' export AIHUB_PW='' aihubshell -mode d -datasetkey 597 -filekey 509338

• https://github.com/SwinTransformer/Video-Swin-Transformer/blob/master/docs/tutorials/3newdataset.md

  • download
    

  ```markdown

  download ### anotation

  Root Root origin train subfolder *.mp4 *.mp4 val *.mp4 label test subfolder *.mp4 *.json customtrainmp4.txt customvalmp4.txt customtestmp4.txt ``` - video_annotion
  -

      ```bash
      python {Download folder}/convert_video_annotation.py
      ```
  
  - train :  val : test = 70 : 15 : 15   
  - videodataset  annotation 
  

  • annotation

    bb_1_210121_two-wheeled-vehicle_236_21840.mp4 206 bb_1_211031_two-wheeled-vehicle_241_21549.mp4 232 bb_1_210125_two-wheeled-vehicle_112_003.mp4 290 bb_1_210917_two-wheeled-vehicle_121_126.mp4 298 ...

Model

• tutorial

  Google Colab Tutorial

1. TSN (optional)

   bash mkdir checkpoints wget -c https://download.openmmlab.com/mmaction/recognition/tsn/tsn_r50_1x1x3_100e_kinetics400_rgb/tsn_r50_1x1x3_100e_kinetics400_rgb_20200614-e508be42.pth \ -O ./checkpoints/tsn_r50_1x1x3_100e_kinetics400_rgb_20200614-e508be42.pth

2. config

   ```python from mmengine import Config import os.path as osp import mmengine from mmengine.runner import Runner from mmengine import Config from mmengine.runner import setrandomseed

   .

   cfg = Config.fromfile('../configs/recognition/tsn/tsnimagenet-pretrained-r508xb32-1x1x3-100e_kinetics400-rgb.py')

   .

   cfg.dataroot = '/mmaction2/data/train/' cfg.datarootval = '/mmaction2/data/val/' cfg.annfiletrain = '/mmaction2/data/customtrainmp4.txt' cfg.annfileval = '/mmaction2/data/customval_mp4.txt'

   .

   cfg.testdataloader.dataset.annfile = '/mmaction2/data/customvalmp4.txt' cfg.testdataloader.dataset.dataprefix.video = '/mmaction2/data/val/'

   .

   cfg.traindataloader.dataset.annfile = '/mmaction2/data/customtrainmp4.txt' cfg.traindataloader.dataset.dataprefix.video = '/mmaction2/data/train/'

   .

   cfg.valdataloader.dataset.annfile = '/mmaction2/data/customvalmp4.txt' cfg.valdataloader.dataset.dataprefix.video = '/mmaction2/data/val/'

   .

   cfg.model.clshead.numclasses = 434

   TSN .

   cfg.loadfrom = './checkpoints/tsnr501x1x3100ekinetics400rgb_20200614-e508be42.pth'

   .

   cfg.workdir = './workspace'

   (LR) 8-GPU .

   1 GPU 8 .

   cfg.traindataloader.batchsize = cfg.traindataloader.batchsize // 16 cfg.valdataloader.batchsize = cfg.valdataloader.batchsize // 16 cfg.optimwrapper.optimizer.lr = cfg.optimwrapper.optimizer.lr / 8 / 16 cfg.traincfg.maxepochs = 50

   .

   cfg.traindataloader.numworkers = 2 cfg.valdataloader.numworkers = 2 cfg.testdataloader.numworkers = 2

   .

   print(f'Config:\n{cfg.pretty_text}')

   .

   mmengine.mkdirorexist(osp.abspath(cfg.work_dir))

   .

   runner = Runner.from_cfg(cfg)

   .

   runner.train()

   .

   runner.test() ```

tester()

```python from mmaction.apis import inferencerecognizer, initrecognizer from mmengine import Config

.

config = './samplework/tsnimagenet-pretrained-r508xb32-1x1x3-100ekinetics400-rgb.py' config = Config.fromfile(config)

.

checkpoint = './samplework/bestacctop1epoch_9.pth'

.

model = init_recognizer(config, checkpoint, device='cuda:0')

.

from operator import itemgetter

testcount = 0 totalcount = 0 with open("../data/customtestmp4.txt", 'r', encoding='utf-8') as file: lines = file.readlines() total_count = len(lines)

for line in lines:
    video_name, video_label = line.split()

    #    
    video = '../data/test/'+video_name
    #   
    label = './index_map.txt'

    #     .
    results = inference_recognizer(model, video)

    #    .
    pred_scores = results.pred_score.tolist()
    #     .
    score_tuples = tuple(zip(range(len(pred_scores)), pred_scores))
    #    .
    score_sorted = sorted(score_tuples, key=itemgetter(1), reverse=True)
    #  5  .
    top5_label = score_sorted[:5]

    #   .
    labels = open(label).readlines()
    #    .
    labels = [x.strip() for x in labels]

    #  5   .
    results = [(labels[k[0]], k[1]) for k in top5_label]

    #  1 
    print(" :"+video_label)
    print(f'{results[0][0]}: ', results[0][1])

    if int(results[0][0]) == int(video_label):
        test_count += 1

print("{}|{} - {}%".format(testcount,totalcount,testcount/totalcount*100)) ```

recognizor()

1. config
   • workspace config
     
2. checkpoint
   • workspace best
     
3. label
   • 0~433, 434 \n /data
     

```python from mmaction.apis import inferencerecognizer, initrecognizer from mmengine import Config

.

config = './samplework/tsnimagenet-pretrained-r508xb32-1x1x8-100ekinetics400-rgb.py' config = Config.fromfile(config)

.

checkpoint = './samplework/bestacctop1epoch_8.pth'

.

model = init_recognizer(config, checkpoint, device='cuda:0')

.

from operator import itemgetter

video = './test2_175.mp4'

label = './index_map.txt'

.

results = inference_recognizer(model, video)

.

predscores = results.predscore.tolist()

.

scoretuples = tuple(zip(range(len(predscores)), pred_scores))

.

scoresorted = sorted(scoretuples, key=itemgetter(1), reverse=True)

5 .

top5label = scoresorted[:5]

.

labels = open(label).readlines()

.

labels = [x.strip() for x in labels]

5 .

results = [(labels[k[0]], k[1]) for k in top5_label]

5 .

print('The top-5 labels with corresponding scores are:') for result in results: print(f'{result[0]}: ', result[1]) ```

1. GPG

   ```bash : GPG error: https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x8664 InRelease: The following signatures couldn't be verified because the public key is not available: NOPUBKEY A4B469963BF863CC :

   NVIDIA CUDA

   RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys A4B469963BF863CC ```

   GPG error: https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x8664 InRelease: The following signatures couldn't be verified because the public key is not available: NOPUBKEY A4B469963BF863CC

2. numpy

   ```bash

   AttributeError: module 'numpy' has no attribute 'int'. np.int was a deprecated alias for the builtin int. To avoid this error in existing code, use int by itself. Doing this will not modify any behavior and is safe. When replacing np.int, you may wish to use e.g. np.int64 or np.int32 to specify the precision. If you wish to review your current use, check the release note link for additional information. The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations ```

   ```bash

   pip install numpy==1.19.0 ```

version

| | | | |------------|-------------|------------------------------------------| |ver 1.0|24.05.26|video-swin-transformer | |ver 1.1|24.05.26|docker file | |ver 1.2|24.05.26|test top5 , | |ver 1.3|24.05.28|otuna | |ver 1.4|24.05.29|bestmodel0529 | |ver 1.5|24.05.31|bestmodel0531 |