https://github.com/ceilingfan456/disr
Science Score: 23.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
○.zenodo.json file
-
○DOI references
-
✓Academic publication links
Links to: arxiv.org, scholar.google -
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (11.4%) to scientific vocabulary
Repository
Basic Info
- Host: GitHub
- Owner: ceilingFan456
- License: apache-2.0
- Language: Python
- Default Branch: main
- Size: 27 MB
Statistics
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
- Releases: 0
Metadata Files
README.md
DiSR-NeRF: Diffusion-Guided View-Consistent Super-Resolution NeRF
Jie Long Lee , Chen Li, Gim Hee Lee
Abstract
We present DiSR-NeRF, a diffusion-guided framework for view-consistent super-resolution (SR) NeRF. Unlike prior works, we circumvent the requirement for high-resolution (HR) reference images by leveraging existing powerful 2D super-resolution models. Nonetheless, independent SR 2D images are often inconsistent across different views. We thus propose Iterative 3D Synchronization (I3DS) to mitigate the inconsistency problem via the inherent multi-view consistency property of NeRF. Specifically, our I3DS alternates between upscaling low-resolution (LR) rendered images with diffusion models, and updating the underlying 3D representation with standard NeRF training. We further introduce Renoised Score Distillation (RSD), a novel score-distillation objective for 2D image resolution. Our RSD combines features from ancestral sampling and Score Distillation Sampling (SDS) to generate sharp images that are also LR-consistent. Qualitative and quantitative results on both synthetic and real-world datasets demonstrate that our DiSR-NeRF can achieve better results on NeRF super-resolution compared with existing works. Code and video results available at the project website.
Installation
``` git clone https://github.com/leejielong/DiSR-NeRF cd DiSR-NeRF
conda create -n disrnerf conda activate disrnerf
Install packages
pip install -r requirements.txt ```
Training
Download NeRF-Synthetic and LLFF datasets here.
Create data directory as follows:
configs
data
├── blender
│ ├── chair
│ └── drums
└── nerf_llff_data
├── fern
└── flower
python launch.py --config configs/nerfdiffusr-sr.yaml --train
Testing
python launch.py --config configs/nerfdiffusr-sr.yaml --test
Citations
@misc{lee2024disrnerf,
title={DiSR-NeRF: Diffusion-Guided View-Consistent Super-Resolution NeRF},
author={Jie Long Lee and Chen Li and Gim Hee Lee},
year={2024},
eprint={2404.00874},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
Acknowledgement
This implementation is built upon threestudio. We thank the authors for the contribution.
Owner
- Name: Huang Qiming
- Login: ceilingFan456
- Kind: user
- Repositories: 1
- Profile: https://github.com/ceilingFan456
GitHub Events
Total
Last Year
Dependencies
- accelerate *
- diffusers ==0.19.3
- einops ==0.6.1
- imageio >=2.28.0
- kornia ==0.7.1
- lightning ==2.0.0
- lpips ==0.1.4
- matplotlib ==3.8.3
- numpy ==1.26.4
- omegaconf ==2.3.0
- opencv-python ==4.6.0.66
- pytorch3d ==0.7.6
- scikit-image ==0.22.0
- scipy ==1.12.0
- tinycudann ==1.7
- torch ==2.0.1
- tqdm ==4.66.2
- transformers ==4.28.1
- trimesh ==3.23.1
- triton ==2.0.0