Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/cloneofsimo/magicmix

Unofficial Implementation of MagicMix

https://github.com/cloneofsimo/magicmix

Science Score: 10.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ○
    codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ✓
    Academic publication links
    Links to: arxiv.org
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (9.8%) to scientific vocabulary

Keywords

diffusion interpolation stable-diffusion
Last synced: 5 months ago · JSON representation

Repository

Unofficial Implementation of MagicMix

Basic Info
Statistics
  • Stars: 98
  • Watchers: 3
  • Forks: 9
  • Open Issues: 3
  • Releases: 0
Topics
diffusion interpolation stable-diffusion
Created over 3 years ago · Last pushed over 3 years ago
Metadata Files
Readme

README.md

MagicMix with Stable Diffusion

Output from this repo's implementation of MagicMix. Original Image, $\nu$ = 0.75, $\nu$ = 0.9 respectively.

Implementation of MagicMix with Stable Diffusion (https://arxiv.org/abs/2210.16056) in PyTorch. This is unofficial Implementation.

Installation

bash pip install git+https://github.com/cloneofsimo/magicmix.git

To get it to work with CUDA GPU, install necessary pytorch and cuda versions.

Explanations

There are three main parameters for MagicMix. $K{min} = k{min ratio} T$, $K{max} = k{max ratio} T$, and $\nu$. $T$ is the number of sampling steps for the scheduler.

magicmix

Basically, $\nu$ determines how little layout image (in the photo above, the corgi) is going to effect the diffusion process. Greater the $\nu$, greater the content is going to effect.

$k{min ratio}$ and $k{max ratio}$ determines the range of the mixing process. If $K{max}$ is large, this will have the same effect as loosing much info of the original layout image. If $K{min}$ is large, this will have the effect of letting content semantic to have effect more freely.

Basic Usage

In the package magic_mix, you can find the implementation of MagicMix with Stable Diffusion. Before running, fill in the variable HF_TOKEN in .env file with Huggingface token for Stable Diffusion, and load_dotenv().

```python from magicmix import magicmixsingleimage

loaddotenv(verbose=True) image = Image.open(inputimage_path).convert("RGB")

mixedsementics = magicmixsingleimage( layoutimage=image, numinferencesteps=50, contentsemanticsprompts=["coffee machine", "tiger"], kmin=20, kmax=30, nu=0.5, guidancescaleatmix=7.5, seed=0, device="cuda:0" ) # mixed sementics is PIL image files...

image[0].save("mixed_sementics.png")

```

Or simply run the following command to generate mixed images.

bash python scripts/run_text_image_mix.py \ --input_image ./examples/inputs/1.jpg \ --output_dir ./examples/outputs \ --num_inference_steps 50 \ --content_semantics_prompts "coffee machine" "tiger" \ --k_min_ratio 0.3 \ --k_max_ratio 0.6 \ --nu 0.5 \ --guidance_scale_at_mix 7.5 \ --seed 0

Owner

  • Name: Simo Ryu
  • Login: cloneofsimo
  • Kind: user
  • Company: Corca AI

Cats are Turing machines cloneofsimo@gmail.com

GitHub Events

Total
  • Watch event: 1
Last Year
  • Watch event: 1

Issues and Pull Requests

Last synced: over 1 year ago

All Time
  • Total issues: 3
  • Total pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Total issue authors: 3
  • Total pull request authors: 0
  • Average comments per issue: 0.33
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • Fangkang515 (1)
  • f8upd8 (1)
  • baymin0220 (1)
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels

Dependencies

requirements.txt pypi
  • diffusers *
  • ftfy *
  • python-dotenv *
  • scipy *
  • transformers *
setup.py pypi
  • for *
  • open *
  • str *