https://github.com/aim-uofa/gendef
Science Score: 23.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
○.zenodo.json file
-
○DOI references
-
✓Academic publication links
Links to: arxiv.org, scholar.google -
○Committers with academic emails
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (10.3%) to scientific vocabulary
Repository
Basic Info
- Host: GitHub
- Owner: aim-uofa
- Default Branch: main
- Size: 8.51 MB
Statistics
- Stars: 39
- Watchers: 5
- Forks: 0
- Open Issues: 0
- Releases: 0
Metadata Files
README.md
GenDeF: Learning Generative Deformation Field for Video Generation
[Wen Wang](https://github.com/encounter1997)1,2* [Kecheng Zheng](https://zkcys001.github.io/)2 [Qiuyu Wang](https://github.com/qiuyu96)2 [Hao Chen](https://scholar.google.com/citations?user=FaOqRpcAAAAJ)1† [Zifan Shi](https://vivianszf.github.io/)3,2* [Ceyuan Yang](https://ceyuan.me/)4[Yujun Shen](https://shenyujun.github.io/)2† [Chunhua Shen](https://cshen.github.io/)1 *Intern at Ant Group †Corresponding Author 1Zhejiang University 2Ant Group 3HKUST 4Shanghai Artificial Intellgence Laboratory
Code will be released soon!
We offer a new perspective on approaching the task of video generation. Instead of directly synthesizing a sequence of frames, we propose to render a video by warping one static image with a generative deformation field (GenDeF). Such a pipeline enjoys three appealing advantages. First, we can sufficiently reuse a well-trained image generator to synthesize the static image (also called canonical image), alleviating the difficulty in producing a video and thereby resulting in better visual quality. Second, we can easily convert a deformation field to optical flows, making it possible to apply explicit structural regularizations for motion modeling, leading to temporally consistent results. Third, the disentanglement between content and motion allows users to process a synthesized video through processing its corresponding static image without any tuning, facilitating many applications like video editing, keypoint tracking, and video segmentation. Both qualitative and quantitative results on three common video generation benchmarks demonstrate the superiority of our GenDeF method.
Main Results
Applications
Video Editing
Point Tracking
Video Segmentation
Diverse Motion Generation
Citing
If you find our work useful, please consider citing:
BibTeX
@misc{wang2023gendef,
title={GenDeF: Learning Generative Deformation Field for Video Generation},
author={Wen Wang and Kecheng Zheng and Qiuyu Wang and Hao Chen and Zifan Shi and Ceyuan Yang and Yujun Shen and Chunhua Shen},
year={2023},
eprint={2312.04561},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
Owner
- Name: Advanced Intelligent Machines (AIM)
- Login: aim-uofa
- Kind: organization
- Location: China
- Repositories: 23
- Profile: https://github.com/aim-uofa
A research team at Zhejiang University, focusing on Computer Vision and broad AI research ...
GitHub Events
Total
- Watch event: 2
Last Year
- Watch event: 2
Committers
Last synced: about 1 year ago
Top Committers
| Name | Commits | |
|---|---|---|
| encounter1997 | 1****0@q****m | 3 |
Committer Domains (Top 20 + Academic)
Issues and Pull Requests
Last synced: about 1 year ago
All Time
- Total issues: 0
- Total pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Total issue authors: 0
- Total pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 0
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 0
- Pull request authors: 0
- Average comments per issue: 0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0