Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/bytedance/imagedream

The code releasing for https://image-dream.github.io/

https://github.com/bytedance/imagedream

Science Score: 33.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ✓
    Academic publication links
    Links to: arxiv.org
  • ✓
    Committers with academic emails
    2 of 26 committers (7.7%) from academic institutions
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (10.0%) to scientific vocabulary

Keywords

research

Keywords from Contributors

diffusion transformers multimodal image-generation image2image stable-diffusion text2image
Last synced: 10 months ago · JSON representation

Repository

The code releasing for https://image-dream.github.io/

Basic Info
  • Host: GitHub
  • Owner: bytedance
  • License: apache-2.0
  • Language: Python
  • Default Branch: main
  • Homepage:
  • Size: 17.3 MB
Statistics
  • Stars: 771
  • Watchers: 9
  • Forks: 36
  • Open Issues: 20
  • Releases: 0
Topics
research
Created over 2 years ago · Last pushed over 2 years ago
Metadata Files
Readme License

README.md

ImageDream Reconstruction

Peng Wang, Yichun Shi

Project Page | Paper | Demo

imagedream-threestudio-teaser

Installation

This part is the same as original MVDream-threestudio. Skip it if you already have installed the environment.

Quickstart

Clone the modelcard on the Huggingface ImageDream Model Page under ./extern/ImageDream/release_models/

In the paper, we use the configuration with soft-shading. It would need an A100 GPU in most cases to compute normal: ```sh export PYTHONPATH=$PYTHONPATH:./extern/ImageDream imagefile="./extern/ImageDream/assets/astronaut.png" ckptfile="./extern/ImageDream/releasemodels/ImageDream/sd-v2.1-base-4view-ipmv.pt" cfgfile="./extern/ImageDream/imagedream/configs/sdv2base_ipmv.yaml"

python3 launch.py \ --config configs/$method.yaml --train --gpu 0 \ name="imagedream-sd21-shading" tag="astronaut" \ system.promptprocessor.prompt="an astronaut riding a horse" \ system.promptprocessor.imagepath="${imagefile}" \ system.guidance.ckptpath="${ckptfile}" \ system.guidance.configpath="${cfgfile}" ```

For diffusion only model, refer to subdir ./extern/ImageDream/ Check ./threestudio/scripts/run_imagedream.sh for a bash example.

Credits

Tips

  1. Place the object in the center and do not make it too large/small in the image.
  2. If you have an object cutting image edge, in config, tuning the parameters range of elevation and fov to be a larger range, e.g. [0, 30], otherwise, you may do image outpainting and follow tips 1.
  3. Check the results with ImageDream diffusion model before using it in 3D rendering to save time.

PreComputed Results

  • Since there is some randomness in diffusion model and time costly to get baseline results. We put our pre-computed results for reproducing Tab.1 in the paper in a hugging face dataset card

Citing

If you find ImageDream helpful, please consider citing:

bibtex @article{wang2023imagedream, title={ImageDream: Image-Prompt Multi-view Diffusion for 3D Generation}, author={Wang, Peng and Shi, Yichun}, journal={arXiv preprint arXiv:2312.02201}, year={2023} }

Owner

  • Name: Bytedance Inc.
  • Login: bytedance
  • Kind: organization
  • Location: Singapore

GitHub Events

Total
  • Issues event: 4
  • Watch event: 58
  • Fork event: 6
Last Year
  • Issues event: 4
  • Watch event: 58
  • Fork event: 6

Committers

Last synced: about 1 year ago

All Time
  • Total Commits: 364
  • Total Committers: 26
  • Avg Commits per committer: 14.0
  • Development Distribution Score (DDS): 0.492
Past Year
  • Commits: 0
  • Committers: 0
  • Avg Commits per committer: 0.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
bennyguo b****o@1****m 185
thuliu-yt16 l****c@g****m 40
Vikram Voleti v****m@i****l 21
Vikram Voleti v****i@g****m 19
Peng wang p****g@b****m 19
Christian Laforte c****e@g****m 13
zouzx n****3@1****m 10
Yichun Shi y****i@b****m 10
Acc-plus 1****0@q****m 8
Ruizhi Shao 2****8@q****m 7
Yanpei Cao c****i@g****m 6
ashawkey a****9@g****m 5
Vikram Voleti v****m@i****l 4
Yosuke Shinya 4****y 3
hbb1 h****b@s****n 2
Jaidev Shriram j****t@g****m 2
Chen Wang c****u@g****m 1
Grace Heseri 3****L 1
Guan Luo 5****1 1
Guanying Chen 4****4@q****m 1
Ikko Eltociear Ashimine e****r@g****m 1
OriginF 5****F 1
Ruilong Li(李瑞龙) r****i@b****u 1
Souhaib Attaiki s****i@g****m 1
Susung Hong j****9@n****m 1
yankeesong 3****g 1
Committer Domains (Top 20 + Academic)
qq.com: 3 bytedance.com: 2 163.com: 2 naver.com: 1 berkeley.edu: 1 shanghaitech.edu.cn: 1 ip-26-0-153-234.us-west-2.compute.internal: 1 ip-26-0-129-82.ec2.internal: 1

Issues and Pull Requests

Last synced: about 1 year ago

All Time
  • Total issues: 21
  • Total pull requests: 1
  • Average time to close issues: N/A
  • Average time to close pull requests: 2 days
  • Total issue authors: 19
  • Total pull request authors: 1
  • Average comments per issue: 0.86
  • Average comments per pull request: 1.0
  • Merged pull requests: 1
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 9
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 8
  • Pull request authors: 0
  • Average comments per issue: 0.11
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • Huangkangxian (2)
  • neciszhang (1)
  • kingback156 (1)
  • knuthnads (1)
  • XuWuLingYu (1)
  • GalDude33 (1)
  • maocaixia (1)
  • anildadj (1)
  • Winjoy (1)
  • LargeRaindrop (1)
  • maximus-victor (1)
  • rabberk (1)
  • qinbaigao (1)
  • yosun (1)
  • furkankrks (1)
Pull Request Authors
  • ashawkey (1)
Top Labels
Issue Labels
Pull Request Labels

Dependencies

.github/workflows/pre-commit.yaml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
docker/Dockerfile docker
  • nvidia/cuda 11.8.0-devel-ubuntu22.04 build
extern/ImageDream/requirements.txt pypi
  • einops *
  • gradio >=3.13.2
  • imageio *
  • imageio-ffmpeg *
  • omegaconf *
  • open-clip-torch ==2.7.0
  • opencv-python *
  • transformers ==4.27.1
  • xformers ==0.0.16
extern/ImageDream/setup.py pypi
  • einops *
  • huggingface_hub *
  • numpy *
  • omegaconf *
  • open-clip-torch *
  • torch *
  • tqdm *
  • transformers *
requirements-dev.txt pypi
  • black * development
  • mypy * development
  • pre-commit * development
  • pylint * development
requirements.txt pypi
  • PyMCubes *
  • accelerate *
  • bitsandbytes *
  • controlnet_aux *
  • diffusers *
  • einops *
  • gradio *
  • huggingface_hub *
  • imageio *
  • imageio >=2.28.0
  • jaxtyping *
  • kornia *
  • libigl *
  • lightning ==2.0.0
  • matplotlib *
  • networkx *
  • omegaconf ==2.3.0
  • open-clip-torch ==2.7.0
  • opencv-python *
  • pysdf *
  • safetensors *
  • sentencepiece *
  • taming-transformers-rom1504 *
  • tensorboard *
  • torchmetrics *
  • transformers *
  • trimesh *
  • typeguard *
  • wandb *
  • xatlas *
  • xformers *