Science Score: 44.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
○DOI references
-
○Academic publication links
-
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (7.9%) to scientific vocabulary
Repository
Basic Info
- Host: GitHub
- Owner: Vencoders
- License: apache-2.0
- Language: Python
- Default Branch: main
- Size: 22.6 MB
Statistics
- Stars: 2
- Watchers: 1
- Forks: 1
- Open Issues: 1
- Releases: 0
Metadata Files
README.md
End-to-end Transformer for Compressed Video Quality Enhancement
Abstract
Convolutional neural networks have achieved excellent results in compressed video quality enhancement task in recent years. State-of-the-art methods explore the spatiotemporal information of adjacent frames mainly by deformable convolution. However, the CNN-based methods can only exploit local information, thus lacking the exploration of global information. Moreover, current methods enhance the video quality at a single scale, ignoring the multi-scale information, which corresponds to information at different receptive fields and is crucial for correlation modeling. Therefore, in this work, we propose a Transformer-based compressed video quality enhancement (TVQE) method, consisting of Transformer based Spatio-Temporal feature Fusion (TSTF) module and Multi-scale Channel-wise Attention based Quality Enhancement (MCQE) module. The proposed TSTF module learns both local and global features for correlation modeling, in which window-based Transformer and the encoder-decoder structure greatly improve the execution efficiency. The proposed MCQE module calculates the multi-scale channel attention, which aggregates the temporal information between channels in the feature map at multiple scales, achieving efficient fusion of inter-frame information. Extensive experiments on the JCT-VT test sequences show that the proposed method increases PSNR by up to 0.98 dB when QP=37. Meanwhile, the inference speed is improved by up to 9.4%, and the number of Flops is reduced by up to 84.4% compared to competing methods at 720p resolution. Moreover, the proposed method achieves the BD-rate reduction up to 23.04%.
Network Architecture
The framework of our proposed TVQE method, which consists of the Transformer based Spatio-Temporal feature Fusion (TSTF) Module and the Multi-scale Channel-wise Attention based Quality Enhancement (MCQE) Module. The TSTF module is designed to exploit spatio-temporal correlation between multiple frames. After TSTF, the multi-scale information between channels in the feature map is further fused by the MCQE module, and finally generate the enhanced frame.
Results
Please check our paper for detail results.
Citation
@article{yu2022end, title={End-to-end Transformer for Compressed Video Quality Enhancement}, author={Yu, Li and Chang, Wenshuai and Wu, Shiyu and Gabbouj, Moncef}, journal={arXiv preprint arXiv:2210.13827}, year={2022} organization={IEEE} }
Owner
- Login: Vencoders
- Kind: user
- Repositories: 2
- Profile: https://github.com/Vencoders
Citation (CITATION.cff)
cff-version: 1.2.0 message: "If you use this software, please cite it as below." authors: - family-names: "Xing" given-names: "Qunliang" orcid: "https://orcid.org/0000-0002-3007-716X" - family-names: "Deng" given-names: "Jianing" title: "PyTorch implementation of STDF" version: 1.0.0 date-released: 2020-9-13 url: "https://github.com/ryanxingql/stdf-pytorch"
GitHub Events
Total
- Issues event: 1
- Push event: 1
- Fork event: 1
Last Year
- Issues event: 1
- Push event: 1
- Fork event: 1
Issues and Pull Requests
Last synced: 10 months ago
All Time
- Total issues: 1
- Total pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Total issue authors: 1
- Total pull request authors: 0
- Average comments per issue: 0.0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 1
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 1
- Pull request authors: 0
- Average comments per issue: 0.0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
- Wjjia0921 (1)