Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/cyberagentailab/layout-dm

[CVPR 2023] LayoutDM: Discrete Diffusion Model for Controllable Layout Generation

https://github.com/cyberagentailab/layout-dm

Science Score: 10.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ○
    codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ✓
    Academic publication links
    Links to: arxiv.org
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (14.9%) to scientific vocabulary

Keywords

cvpr2023 diffusion-models generative-ai layout pytorch
Last synced: 5 months ago · JSON representation

Repository

[CVPR 2023] LayoutDM: Discrete Diffusion Model for Controllable Layout Generation

Basic Info
Statistics
  • Stars: 252
  • Watchers: 3
  • Forks: 27
  • Open Issues: 3
  • Releases: 0
Topics
cvpr2023 diffusion-models generative-ai layout pytorch
Created almost 3 years ago · Last pushed over 2 years ago
Metadata Files
Readme License

README.md

LayoutDM: Discrete Diffusion Model for Controllable Layout Generation (CVPR2023)

This repository is an official implementation of the paper titled above. Please refer to project page or paper for more details.

Setup

Here we describe the setup required for the model training and evaluation.

Requirements

We check the reproducibility under this environment. - Python3.7 - CUDA 11.3 - PyTorch 1.10

We recommend using Poetry (all settings and dependencies in pyproject.toml). Pytorch-geometry provides independent pre-build wheel for a combination of PyTorch and CUDA version (see PyG:Installation for details). If your environment does not match the one above, please update the dependencies.

How to install

  1. Install poetry (see official docs). We recommend to make a virtualenv and install poetry inside it.

bash curl -sSL https://install.python-poetry.org | python3 -

  1. Install dependencies (it may be slow..)

bash poetry install

  1. Download resources and unzip

bash wget https://github.com/CyberAgentAILab/layout-dm/releases/download/v1.0.0/layoutdm_starter.zip unzip layoutdm_starter.zip

The data is decompressed to the following structure: download - clustering_weights - datasets - fid_weights - pretrained_weights

Experiment

Important: we find some critical errors that cannot be fixed quickly in using multiple GPUs. Please set CUDA_VISIBLE_DEVICES=<GPU_ID> to force the model use a single GPU.

Note: our main framework is based on hydra. It is convenient to handle dozens of arguments hierarchically but may require some additional efforts if one is new to hydra.

Demo

Please run a jupyter notebook in notebooks/demo.ipynb. You can get and render the results of six layout generation tasks on two datasets (Rico and PubLayNet).

Training

You can also train your own model from scratch, for example by

bash bash bin/train.sh rico25 layoutdm

, where the first and second argument specifies the dataset (choices) and the type of experiment (choices), respectively. Note that for training/testing, style of the arguments is key=value because we use hydra, unlike popular --key value (e.g., argparse).

Testing

bash poetry run python3 -m src.trainer.trainer.test \ cond=<COND> \ job_dir=<JOB_DIR> \ result_dir=<RESULT_DIR> \ <ADDITIONAL_ARGS> <COND> can be: (unconditional, c, cwh, partial, refinement, relation)

For example, if you want to test the provided LayoutDM model on C->S+P, the command is as follows: poetry run python3 -m src.trainer.trainer.test cond=c dataset_dir=./download/datasets job_dir=./download/pretrained_weights/layoutdm_rico result_dir=tmp/dummy_results

Please refer to TestConfig for more options available. Below are some popular options for - is_validation=true: used to evaluate the generation performance on validation set instead of test set. This must be used when tuning the hyper-parameters. - sampling=top_p top_p=<TOP_P>: use top-p sampling with p=　instead of default sampling.

Evaluation

bash poetry run python3 eval.py <RESULT_DIR>

Citation

If you find this code useful for your research, please cite our paper:

@inproceedings{inoue2023layout, title={{LayoutDM: Discrete Diffusion Model for Controllable Layout Generation}}, author={Naoto Inoue and Kotaro Kikuchi and Edgar Simo-Serra and Mayu Otani and Kota Yamaguchi}, booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, year={2023}, pages={10167-10176}, }

Owner

  • Name: CyberAgent AI Lab
  • Login: CyberAgentAILab
  • Kind: organization
  • Location: Japan

GitHub Events

Total
  • Issues event: 2
  • Watch event: 52
  • Fork event: 5
Last Year
  • Issues event: 2
  • Watch event: 52
  • Fork event: 5

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 1
  • Total pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Total issue authors: 1
  • Total pull request authors: 0
  • Average comments per issue: 0.0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 1
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 1
  • Pull request authors: 0
  • Average comments per issue: 0.0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • Supercarry-khr (3)
  • hsinwu13 (1)
  • Usernamezhx (1)
  • aBu-ctrl (1)
  • liuan0803 (1)
  • billy322 (1)
  • andupotorac (1)
  • Melon-Xu (1)
  • souvikg544 (1)
  • dips4717 (1)
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels

Packages

  • Total packages: 2
  • Total downloads: unknown
  • Total dependent packages: 0
    (may contain duplicates)
  • Total dependent repositories: 0
    (may contain duplicates)
  • Total versions: 2
proxy.golang.org: github.com/cyberagentailab/layout-dm
  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent packages count: 6.5%
Average: 6.7%
Dependent repos count: 7.0%
Last synced: 6 months ago
proxy.golang.org: github.com/CyberAgentAILab/layout-dm
  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
Rankings
Dependent packages count: 6.5%
Average: 6.7%
Dependent repos count: 7.0%
Last synced: 6 months ago

Dependencies

poetry.lock pypi
  • 148 dependencies
pyproject.toml pypi
  • pysen ^0.10.1 develop
  • pytest ^7.0 develop
  • Cython ^0.29.30
  • Pillow ^9.1.0
  • einops ^0.4.1
  • fsspec 2023.1.0
  • gcsfs 2023.1.0
  • google-cloud-storage ^2.2.1
  • hydra-core ^1.1.2
  • ipykernel ^6.13.0
  • ipython <7.32.0
  • matplotlib 3.5.2
  • numpy 1.21.6
  • pandas 1.3.5
  • prdc ^0.2
  • pycocotools ^2.0.4
  • python ^3.7.1,<3.8
  • pytorch-fid ^0.2.1
  • scikit-learn 1.0.2
  • scipy >=1.4,<1.10
  • seaborn ^0.11.2
  • setuptools 59.5.0
  • tensorflow ^2.8.0
  • torch --- - !ruby/hash:ActiveSupport::HashWithIndifferentAccess url: https://download.pytorch.org/whl/cu113/torch-1.10.2%2Bcu113-cp37-cp37m-linux_x86_64.whl platform: linux
  • torch-geometric ^2.0.4
  • torch-scatter *
  • torch-sparse *
  • torch-tb-profiler ^0.4.0
  • torchvision *
  • tqdm ^4.64.0