https://github.com/ai-forever/movqgan
MoVQGAN - model for the image encoding and reconstruction
Science Score: 10.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
○codemeta.json file
-
○.zenodo.json file
-
○DOI references
-
✓Academic publication links
Links to: arxiv.org -
○Committers with academic emails
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (7.0%) to scientific vocabulary
Keywords
Repository
MoVQGAN - model for the image encoding and reconstruction
Basic Info
Statistics
- Stars: 233
- Watchers: 4
- Forks: 16
- Open Issues: 10
- Releases: 0
Topics
Metadata Files
README.md
SBER-MoVQGAN
SBER-MoVQGAN (Modulated Vector Quantized GAN) is a new SOTA model in the image reconstruction problem. This model is based on code from the VQGAN repository and modifications from the original MoVQGAN paper. The architecture of SBER-MoVQGAN is shown below in the figure.
SBER-MoVQGAN was successfully implemented in Kandinsky 2.1, and became one of the architecture blocks that allowed to significantly improve the quality of image generation from text.
Models
The following table shows a comparison of the models on the Imagenet dataset in terms of FID, SSIM, and PSNR metrics. A more detailed description of the experiments and a comparison with other models can be found in the Habr post.
|Model|Latent size|Num Z|Train steps|FID|SSIM|PSNR|L1| |:----|:----|:----|:----|:----|:----|:----|:----| |ViT-VQGAN*|32x32|8192|500000|1,28|-|-|-| |RQ-VAE*|8x8x16|16384|10 epochs|1,83|-|-|-| |Mo-VQGAN*|16x16x4|1024|40 epochs|1,12|0,673|22,42|-| | VQ CompVis| 32x32| 16384 | 971043| 1,34| 0,65| 23,847| 0,053| | KL CompVis| 32x32| - | 246803| 0,968| 0,692| 25,112| 0,047| | SBER-VQGAN (from pretrain)| 32x32| 8192| 1 epoch| 1,439| 0,682| 24,314| 0,05| | SBER-MoVQGAN 67M | 32x32 | 16384 | 2M | 0,965| 0,725| 26,449| 0,042 | SBER-MoVQGAN 102M|32x32|16384|2360k|0,776|0,737 | 26,889| 0,04| |SBER-MoVQGAN 270M|32x32|16384|1330k| 0,686💥| 0,741💥| 27,037💥| 0,039💥|
How to use
Install
pip install "git+https://github.com/ai-forever/MoVQGAN.git"
Train
python main.py --config configs/movqgan_270M.yaml
Inference
Check jupyter notebook with example in ./notebooks folder or
Examples
This section provides examples of image reconstruction for all versions of SBER-MoVQGAN on hard-to-recover domains such as faces, text, and other complex scenes.
Authors
Owner
- Name: AI Forever
- Login: ai-forever
- Kind: organization
- Location: Armenia
- Repositories: 60
- Profile: https://github.com/ai-forever
Creating ML for the future. AI projects you already know. We are non-profit organization with members from all over the world.
GitHub Events
Total
- Issues event: 2
- Watch event: 98
- Fork event: 5
Last Year
- Issues event: 2
- Watch event: 98
- Fork event: 5
Committers
Last synced: about 1 year ago
Top Committers
| Name | Commits | |
|---|---|---|
| NastyaMittseva | n****a@m****u | 54 |
Committer Domains (Top 20 + Academic)
Issues and Pull Requests
Last synced: 10 months ago
All Time
- Total issues: 9
- Total pull requests: 1
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Total issue authors: 9
- Total pull request authors: 1
- Average comments per issue: 0.44
- Average comments per pull request: 1.0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Past Year
- Issues: 3
- Pull requests: 0
- Average time to close issues: N/A
- Average time to close pull requests: N/A
- Issue authors: 3
- Pull request authors: 0
- Average comments per issue: 0.0
- Average comments per pull request: 0
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 0
Top Authors
Issue Authors
- Sun-Made-By-Yi (1)
- minkowski0125 (1)
- bladewaltz1 (1)
- Droliven (1)
- tengjiayan20 (1)
- UESTC-Med424-JYX (1)
- Tsai-chia-hsiang (1)
- lagokun (1)