flux-controlnet-inpaint

https://github.com/fulfulggg/flux-controlnet-inpaint

Science Score: 44.0%

This score indicates how likely this project is to be science-related based on various indicators:

✓
CITATION.cff file
Found CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
○
DOI references
○
Academic publication links
○
Academic email domains
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (8.0%) to scientific vocabulary

Last synced: 6 months ago · JSON representation ·

Repository

Basic Info

Host: GitHub
Owner: fulfulggg
License: apache-2.0
Language: Python
Default Branch: main
Size: 56 MB

Statistics

Stars: 0
Watchers: 1
Forks: 0
Open Issues: 0
Releases: 0

Created over 1 year ago · Last pushed about 1 year ago

Metadata Files

Readme Contributing License Code of conduct Citation

Introduction

中文版本 This project mainly introduces how to combine flux and controlnet for inpaint, taking the children's clothing scene as an example. For more detailed introduction, please refer to the third section of yishaoai/tutorials-of-100-wonderful-ai-models.

The usage of this project is as follows:

```python import torch from diffusers.utils import loadimage from diffusers.pipelines.flux.pipelinefluxcontrolnetinpaint import FluxControlNetInpaintPipeline from diffusers.models.controlnetflux import FluxControlNetModel from controlnetaux import CannyDetector

basemodel = 'black-forest-labs/FLUX.1-dev' controlnetmodel = 'YishaoAI/flux-dev-controlnet-canny-kid-clothes'

pipe = FluxControlNetInpaintPipeline.frompretrained(basemodel, controlnet=controlnet, torchdtype=torch.bfloat16) pipe.enablemodelcpuoffload() #save some VRAM by offloading the model to CPU. Remove this if you have enough GPU power pipe.to("cuda")

image = loadimage(imagepath) mask = loadimage(maskpath) canny = CannyDetector() canny_image = canny(image) prompt = "children's clothing model"

imageres = pipe( prompt, image=image, controlimage=cannyimage, controlnetconditioningscale=0.5, maskimage=mask, strength=0.95, numinferencesteps=50, guidancescale=5, generator=generator, jointattention_kwargs={"scale": 0.8}, ).images[0] ```

Result example

The following example image is based on the children's clothing scene. The clothing part will remain unchanged, and the portrait and background will be redrawn based on the prompt word. Because controlnet is used, the edges of these images will be similar. The weight of controlnet can be adjusted based on the need. The larger the weight, the more edge information of the generated image is retained, and the smaller the weight, the less edge information is retained. The inpaint method is suitable for reproducing popular products, which is very suitable for the needs of buyers' shows and users in the e-commerce field. At the same time, it can also provide merchants with the main picture of clothing models.

demo

Owner

Login: fulfulggg
Kind: user

Repositories: 1
Profile: https://github.com/fulfulggg

Citation (CITATION.cff)

cff-version: 1.2.0
title: 'Diffusers: State-of-the-art diffusion models'
message: >-
  If you use this software, please cite it using the
  metadata from this file.
type: software
authors:
  - given-names: Patrick
    family-names: von Platen
  - given-names: Suraj
    family-names: Patil
  - given-names: Anton
    family-names: Lozhkov
  - given-names: Pedro
    family-names: Cuenca
  - given-names: Nathan
    family-names: Lambert
  - given-names: Kashif
    family-names: Rasul
  - given-names: Mishig
    family-names: Davaadorj
  - given-names: Dhruv
    family-names: Nair
  - given-names: Sayak
    family-names: Paul
  - given-names: Steven
    family-names: Liu
  - given-names: William
    family-names: Berman
  - given-names: Yiyi
    family-names: Xu
  - given-names: Thomas
    family-names: Wolf
repository-code: 'https://github.com/huggingface/diffusers'
abstract: >-
  Diffusers provides pretrained diffusion models across
  multiple modalities, such as vision and audio, and serves
  as a modular toolbox for inference and training of
  diffusion models.
keywords:
  - deep-learning
  - pytorch
  - image-generation
  - hacktoberfest
  - diffusion
  - text2image
  - image2image
  - score-based-generative-modeling
  - stable-diffusion
  - stable-diffusion-diffusers
license: Apache-2.0
version: 0.12.1

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Open Source Science