Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/cyberagentailab/opencole

OpenCOLE: Towards Reproducible Automatic Graphic Design Generation [Inoue+, CVPRW2024 (GDUG)]

https://github.com/cyberagentailab/opencole

Science Score: 23.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ✓
    Academic publication links
    Links to: arxiv.org, scholar.google
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (12.2%) to scientific vocabulary
Last synced: 10 months ago · JSON representation

Repository

OpenCOLE: Towards Reproducible Automatic Graphic Design Generation [Inoue+, CVPRW2024 (GDUG)]

Basic Info
  • Host: GitHub
  • Owner: CyberAgentAILab
  • License: apache-2.0
  • Language: Python
  • Default Branch: main
  • Homepage:
  • Size: 6.72 MB
Statistics
  • Stars: 62
  • Watchers: 4
  • Forks: 7
  • Open Issues: 4
  • Releases: 0
Created about 2 years ago · Last pushed over 1 year ago
Metadata Files
Readme License

README.md

OpenCOLE: Towards Reproducible Automatic Graphic Design Generation

Naoto Inoue*  Kento Masui*  Wataru Shimoda*  Kota Yamaguchi  (*: equal contribution)

CyberAgent

Workshop on Graphic Design Understanding and Generation (at CVPR2024)

![alt text](figs/main_results.png)

Overview

🤔 Automatic generation of graphic designs has recently received considerable attention.

😦 However, the state-of-the-art approaches are complex and rely on proprietary datasets, which creates reproducibility barriers.

🔥 In this paper, we propose an open framework for automatic graphic design called OpenCOLE, where we build a modified version of the pioneering COLE [Jia+, arXiv'23] and train our model exclusively on publicly available datasets.

🚀 Based on GPT4V evaluations, our model shows promising performance comparable to the original COLE. We release the pipeline and training results to encourage open development.

Setup

Requirements

Install

poetry install

Dataset

OpenCOLE dataset (v1) is available at cyberagent/opencole in HuggingFace dataset hub.

Pre-trained models

Environment variables

Some part requires additional environment variables. We recommend to use direnv. Please copy the template in .envrc.example and modify it on your own.

bash cp .envrc.example .envrc

Inference

Please refer to inference.md.

Evaluation

We provide a script for GPT4V-based evaluation on generated images.

python uv run python -m opencole.evaluation.eval_gpt4v --input_dir <INPUT_DIR> --output_path <OUTPUT_PATH>

Training

Please refer to training.md.

Citation

If you find this code useful for your research, please cite our paper:

@inproceedings{inoue2024opencole, title={{OpenCOLE: Towards Reproducible Automatic Graphic Design Generation}}, author={Naoto Inoue and Kento Masui and Wataru Shimoda and Kota Yamaguchi}, booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)}, year={2024}, }

Acknowledgement

This repository has been migrated from the internal repo. Despite the fact that commit logs are not visible, all the contributors have made significant contributions to the repository.

Owner

  • Name: CyberAgent AI Lab
  • Login: CyberAgentAILab
  • Kind: organization
  • Location: Japan

GitHub Events

Total
  • Issues event: 7
  • Watch event: 33
  • Delete event: 5
  • Issue comment event: 5
  • Push event: 8
  • Pull request event: 9
  • Fork event: 8
  • Create event: 4
Last Year
  • Issues event: 7
  • Watch event: 33
  • Delete event: 5
  • Issue comment event: 5
  • Push event: 8
  • Pull request event: 9
  • Fork event: 8
  • Create event: 4

Dependencies

pyproject.toml pypi
  • ipykernel * develop
  • ipython * develop
  • mypy ^1 develop
  • pytest * develop
  • ruff * develop
  • tensorboard * develop
  • bitsandbytes *
  • chromadb *
  • compel *
  • datasets *
  • diffusers *
  • faiss-cpu *
  • huggingface-hub *
  • langchain *
  • langchain-openai *
  • matplotlib *
  • openai *
  • peft *
  • pinjected *
  • protobuf *
  • pydantic >=2.5.2
  • python >=3.10,<3.13
  • scikit-image *
  • scikit-learn *
  • seaborn *
  • sentence-transformers *
  • sentencepiece *
  • setuptools *
  • skia-python *
  • torch --- - !ruby/hash:ActiveSupport::HashWithIndifferentAccess version: "^2.2.1" platform: darwin source: pypi - !ruby/hash:ActiveSupport::HashWithIndifferentAccess version: "^2.2.1+cu121" platform: linux source: torch_cu121
  • torchvision --- - !ruby/hash:ActiveSupport::HashWithIndifferentAccess version: "^0.17.1" platform: darwin source: pypi - !ruby/hash:ActiveSupport::HashWithIndifferentAccess version: "^0.17.1+cu121" platform: linux source: torch_cu121
  • transformers 4.37.2