BasicVSR_PlusPlus (CVPR 2022)

[Paper] [Project Page] [Code]

This is the official repository for BasicVSR++. Please feel free to raise issue related to BasicVSR++! If you are also interested in RealBasicVSR, which is also accepted to CVPR 2022, please don't hesitate to star!

Authors: Kelvin C.K. Chan, Shangchen Zhou, Xiangyu Xu, Chen Change Loy, Nanyang Technological University

Acknowedgement: Our work is built upon MMEditing. Please follow and star this repository and MMEditing!

News

• 2 Dec 2021: Colab demo released
• 18 Apr 2022: Code released. Also merged into MMEditing
• 5 Feb 2023: The checkpoints for BasicVSR_2x is released.

TODO

• [ ] Add BasicVSR_2x architecture
• [x] ~~Add BasicVSR_2x checkpoints~~
• [ ] Add data processing scripts
• [x] ~~Add checkpoints for deblur and denoise~~
• [x] ~~Add configs for deblur and denoise~~
• [x] ~~Add Colab demo~~

Pre-trained Weights

You can find the pre-trained weights for deblurring and denoising in this link. For super-resolution and compressed video enhancement, please refer to MMEditing.

Installation

1. Install PyTorch
2. pip install openmim
3. mim install mmcv-full
4. git clone https://github.com/ckkelvinchan/BasicVSR_PlusPlus.git
5. cd BasicVSR_PlusPlus
6. pip install -v -e .

Inference a Video

1. Download pre-trained weights
2. python demo/restoration_video_demo.py ${CONFIG} ${CHKPT} ${IN_PATH} ${OUT_PATH}

For example, you can download the VSR checkpoint here to chkpts/basicvsr_plusplus_reds4.pth, then run python demo/restoration_video_demo.py configs/basicvsr_plusplus_reds4.py chkpts/basicvsr_plusplus_reds4.pth data/demo_000 results/demo_000 You can also replace ${IN_PATH} ${OUT_PATH} by your video path (e.g., xxx/yyy.mp4) to input/output videos.

Training Models

1. Put the dataset in the designated locations specified in the configuration file.
2. sh tools/dist_train.sh ${CONFIG} ${NGPUS}

Data Preprocessing

To be added...

Related Work

Our BasicVSR series: 1. BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond, CVPR 2021 2. Investigating Tradeoffs in Real-World Video Super-Resolution, CVPR 2022

More about deformable alignment: - Understanding Deformable Alignment in Video Super-Resolution, AAAI 2021

Citations

@inproceedings{chan2022basicvsrpp, author = {Chan, Kelvin C.K. and Zhou, Shangchen and Xu, Xiangyu and Loy, Chen Change}, title = {{BasicVSR++}: Improving video super-resolution with enhanced propagation and alignment}, booktitle = {IEEE Conference on Computer Vision and Pattern Recognition}, year = {2022} } @article{chan2022generalization, title={On the Generalization of {BasicVSR++} to Video Deblurring and Denoising}, author={Chan, Kelvin CK and Zhou, Shangchen and Xu, Xiangyu and Loy, Chen Change}, journal={arXiv preprint arXiv:2204.05308}, year={2022} }