fairdiffusion

[Science Advances] FairDiffusion: Enhancing Equity in Latent Diffusion Models via Fair Bayesian Perturbation

https://github.com/harvard-ophthalmology-ai-lab/fairdiffusion

Science Score: 44.0%

This score indicates how likely this project is to be science-related based on various indicators:

✓
CITATION.cff file
Found CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
○
DOI references
○
Academic publication links
○
Academic email domains
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (11.1%) to scientific vocabulary

Last synced: 10 months ago · JSON representation ·

Repository

[Science Advances] FairDiffusion: Enhancing Equity in Latent Diffusion Models via Fair Bayesian Perturbation

Basic Info

Host: GitHub
Owner: Harvard-Ophthalmology-AI-Lab
License: apache-2.0
Language: Python
Default Branch: main
Homepage: https://www.science.org/doi/full/10.1126/sciadv.ads4593
Size: 3.63 MB

Statistics

Stars: 3
Watchers: 2
Forks: 1
Open Issues: 0
Releases: 0

Created almost 2 years ago · Last pushed about 1 year ago

Metadata Files

Readme Contributing License Code of conduct Citation

FairDiffusion

Despite the strong performance of these generative models, it remains an open question whether the quality of image generation is consistent across different demographic subgroups. To tackle these biases, we introduce FairDiffusion, an equity-aware latent diffusion model that enhances fairness in medical image generation via Fair Bayesian Perturbation.

Requirements

To install the prerequisites, run: conda env create -f environment.yml or pip install -r requirements.txt

FairGenMed Dataset

We present FairGenMed, the first dataset for studying fairness of medical generative models, providing detailed quantitative measurements of multiple clinical conditions to investigate the semantic correlation between text prompts and anatomical regions across various demographic subgroups.

The dataset can be accessed via this link. This dataset can only be used for non-commercial research purposes. At no time, the dataset shall be used for clinical decisions or patient care. The data use license is CC BY-NC-ND 4.0. If you have any questions, please email harvardophai@gmail.com and harvardairobotics@gmail.com.

Note that, the modifier word “Harvard” only indicates that our dataset is from the Department of Ophthalmology of Harvard Medical School and does not imply an endorsement, sponsorship, or assumption of responsibility by either Harvard University or Harvard Medical School as a legal identity.

Our dataset includes 10,000 subjects for glaucoma detection with comprehensive demographic identity attributes including age, gender, race, ethnicity, preferred language, and marital status. Each subject has one Scanning Laser Ophthalmoscopy (SLO) fundus photo and one npz file. The size of SLO fundus photos is 512 x 664.

The NPZ files have the following attributes glaucoma: the label of glaucoma disease, 0 - non-glaucoma, 1 - glaucoma oct_bscans: images of OCT B-scans race: 0 - Asian, 1 - Black, 2 - White male: 0 - Female, 1 - Male hispanic: 0 - Non-Hispanic, 1 - Hispanic maritalstatus: 0 - Married or Partnered, 1 - Single, 2 - Divorced, 3 - Widowed, 4 - Legally Separated, and -1 - Unknown language: 0 - English, 1 - Spanish, 2 - Others

We have compiled all clinical measurements related to the 10,000 samples into a meta CSV file named data_summary.csv. Specifically, the cup-disc ratio, severity of vision loss, and status of spherical equivalent are denoted by the column names 'cdrstatus', 'mdseverity', and 'sestatus', respectively, in the datasummary.csv file.

Experiments

Train Stable Diffusion on FairGenMed Dataset cd examples/text_to_image export MODEL_NAME="stabilityai/stable-diffusion-2-1" accelerate launch --main_process_port 29601 --multi_gpu --num_processes 2 --mixed_precision="fp16" train_text_to_image.py --text_encoder_type clip --dataset_dir <DATASET_DIR> --checkpointing_steps 5000 --pretrained_model_name_or_path=$MODEL_NAME --train_data_dir tmpp --datasets glaucoma --race_prompt --gender_prompt --ethnicity_prompt --use_ema --resolution=512 --center_crop --random_flip --train_batch_size=16 --gradient_accumulation_steps=1 --gradient_checkpointing --max_train_steps=100000 --learning_rate=1e-05 --max_grad_norm=1 --lr_scheduler="constant" --lr_warmup_steps=0 --output_dir=<OUTPUT_DIR> --cache_dir <CACHE_DIR>

Train Stable Diffusion on HAM10000 Dataset cd examples/text_to_image export MODEL_NAME="stabilityai/stable-diffusion-2-1" accelerate launch --main_process_port 29601 --multi_gpu --num_processes 2 --mixed_precision="fp16" train_text_to_image_ham.py --text_encoder_type clip --dataset_dir <DATASET_DIR> --checkpointing_steps 5000 --pretrained_model_name_or_path=$MODEL_NAME --train_data_dir tmpp --datasets glaucoma --race_prompt --gender_prompt --ethnicity_prompt --use_ema --resolution=512 --center_crop --random_flip --train_batch_size=16 --gradient_accumulation_steps=1 --gradient_checkpointing --max_train_steps=100000 --learning_rate=1e-05 --max_grad_norm=1 --lr_scheduler="constant" --lr_warmup_steps=0 --output_dir=<OUTPUT_DIR> --cache_dir <CACHE_DIR>

Train Stable Diffusion on Chexpert Dataset cd examples/text_to_image export MODEL_NAME="stabilityai/stable-diffusion-2-1" accelerate launch --main_process_port 29601 --multi_gpu --num_processes 2 --mixed_precision="fp16" train_text_to_image_chexpert.py --text_encoder_type clip --dataset_dir <DATASET_DIR> --checkpointing_steps 5000 --pretrained_model_name_or_path=$MODEL_NAME --train_data_dir tmpp --datasets glaucoma --race_prompt --gender_prompt --ethnicity_prompt --use_ema --resolution=512 --center_crop --random_flip --train_batch_size=16 --gradient_accumulation_steps=1 --gradient_checkpointing --max_train_steps=100000 --learning_rate=1e-05 --max_grad_norm=1 --lr_scheduler="constant" --lr_warmup_steps=0 --output_dir=<OUTPUT_DIR> --cache_dir <CACHE_DIR>

Train FairDiffusion on FairGenMed Dataset cd examples/text_to_image export MODEL_NAME="stabilityai/fair-diffusion-2-1" export TIME_WINDOW=30 export EXPLOITATION=0.95 accelerate launch --main_process_port 29601 --multi_gpu --num_processes 2 --mixed_precision="fp16" train_text_to_image_proposed.py --text_encoder_type clip --dataset_dir <DATASET_DIR> --checkpointing_steps 5000 --pretrained_model_name_or_path=$MODEL_NAME --train_data_dir tmpp --datasets glaucoma --race_prompt --gender_prompt --ethnicity_prompt --use_ema --resolution=512 --center_crop --random_flip --train_batch_size=16 --gradient_accumulation_steps=1 --gradient_checkpointing --max_train_steps=100000 --learning_rate=1e-05 --max_grad_norm=1 --lr_scheduler="constant" --lr_warmup_steps=0 --output_dir=<OUTPUT_DIR> --cache_dir <CACHE_DIR> --fair_time_window ${TIME_WINDOW} --fair_exploitation_rate ${EXPLOITATION}

Train FairDiffusion on HAM10000 Dataset cd examples/text_to_image export MODEL_NAME="stabilityai/fair-diffusion-2-1" export TIME_WINDOW=30 export EXPLOITATION=0.95 accelerate launch --main_process_port 29601 --multi_gpu --num_processes 2 --mixed_precision="fp16" train_text_to_image_proposed_ham.py --text_encoder_type clip --dataset_dir <DATASET_DIR> --checkpointing_steps 5000 --pretrained_model_name_or_path=$MODEL_NAME --train_data_dir tmpp --datasets glaucoma --race_prompt --gender_prompt --ethnicity_prompt --use_ema --resolution=512 --center_crop --random_flip --train_batch_size=16 --gradient_accumulation_steps=1 --gradient_checkpointing --max_train_steps=100000 --learning_rate=1e-05 --max_grad_norm=1 --lr_scheduler="constant" --lr_warmup_steps=0 --output_dir=<OUTPUT_DIR> --cache_dir <CACHE_DIR> --fair_time_window ${TIME_WINDOW} --fair_exploitation_rate ${EXPLOITATION}

Train FairDiffusion on Chexpert Dataset cd examples/text_to_image export MODEL_NAME="stabilityai/fair-diffusion-2-1" export TIME_WINDOW=30 export EXPLOITATION=0.95 accelerate launch --main_process_port 29601 --multi_gpu --num_processes 2 --mixed_precision="fp16" train_text_to_image_proposed_chexpert.py --text_encoder_type clip --dataset_dir <DATASET_DIR> --checkpointing_steps 5000 --pretrained_model_name_or_path=$MODEL_NAME --train_data_dir tmpp --datasets glaucoma --race_prompt --gender_prompt --ethnicity_prompt --use_ema --resolution=512 --center_crop --random_flip --train_batch_size=16 --gradient_accumulation_steps=1 --gradient_checkpointing --max_train_steps=100000 --learning_rate=1e-05 --max_grad_norm=1 --lr_scheduler="constant" --lr_warmup_steps=0 --output_dir=<OUTPUT_DIR> --cache_dir <CACHE_DIR> --fair_time_window ${TIME_WINDOW} --fair_exploitation_rate ${EXPLOITATION}

Visualize Diffusion Model (Generation) python visualize_fairdiffusion.py --dataset_dir <DATASET_DIR> --initial_model stabilityai/stable-diffusion-2-1 --model_path <OUTPUT_DIR>/checkpoint-<TBD>/unet --datasets glaucoma --race_prompt --gender_prompt --ethnicity_prompt --vis_dir <VIS_DIR> --prompts prompts.txt --repeat_prompt 5

Evaluate Fairness of Diffusion Model (Generation) python evaluate_fairdiffusion.py --metrics_calculation_idx 1 --dataset_dir <DATASET_DIR> --initial_model stabilityai/stable-diffusion-2-1 --model_path <OUTPUT_DIR>/checkpoint-<TBD>/unet --datasets glaucoma --race_prompt --gender_prompt --ethnicity_prompt > OUTPUT.txt

Evaluate Fairness of Diffusion Model (Classification) - ViT-B ``` cd classificationcodebase DATASETDIR= RESULTDIR= MODELTYPE=ViT-B MODALITYTYPE=slofundus

VITWEIGHTS=imagenet BATCHSIZE=64 BLR=5e-4 WD=0.01 LD=0.55 DP=0.1

Baselines

EXPNAME=BASELINEGLAUCOMA PERFFILE=${MODELTYPE}${MODALITYTYPE}${EXPNAME}.csv python scripts/trainglaucomafair.py --task cls --epochs 50 --batchsize ${BATCHSIZE} --blr ${BLR} --minlr 1e-6 --warmupepochs 5 --weightdecay ${WD} --layerdecay ${LD} --droppath ${DP} --datadir ${DATASETDIR}/ --resultdir ${RESULTDIR}/${MODELTYPE}${MODALITYTYPE}${EXPNAME} --modeltype ${MODELTYPE} --modalitytypes ${MODALITYTYPE} --perffile ${PERFFILE} --vitweights ${VITWEIGHTS}

EXPNAME=BASELINEMDSEVERITY PERFFILE=${MODELTYPE}${MODALITYTYPE}${EXPNAME}.csv python scripts/trainglaucomafair.py --task mdseverity --epochs 50 --batchsize ${BATCHSIZE} --blr ${BLR} --minlr 1e-6 --warmupepochs 5 --weightdecay ${WD} --layerdecay ${LD} --droppath ${DP} --datadir ${DATASETDIR}/ --resultdir ${RESULTDIR}/${MODELTYPE}${MODALITYTYPE}${EXPNAME} --modeltype ${MODELTYPE} --modalitytypes ${MODALITYTYPE} --perffile ${PERFFILE} --vitweights ${VITWEIGHTS}

EXPNAME=BASELINECDRSTATUS PERFFILE=${MODELTYPE}${MODALITYTYPE}${EXPNAME}.csv python scripts/trainglaucomafair.py --task cdrstatus --epochs 50 --batchsize ${BATCHSIZE} --blr ${BLR} --minlr 1e-6 --warmupepochs 5 --weightdecay ${WD} --layerdecay ${LD} --droppath ${DP} --datadir ${DATASETDIR}/ --resultdir ${RESULTDIR}/${MODELTYPE}${MODALITYTYPE}${EXPNAME} --modeltype ${MODELTYPE} --modalitytypes ${MODALITYTYPE} --perffile ${PERFFILE} --vitweights ${VITWEIGHTS}

EXPNAME=BASELINESESTATUS PERFFILE=${MODELTYPE}${MODALITYTYPE}${EXPNAME}.csv python scripts/trainglaucomafair.py --task sestatus --epochs 50 --batchsize ${BATCHSIZE} --blr ${BLR} --minlr 1e-6 --warmupepochs 5 --weightdecay ${WD} --layerdecay ${LD} --droppath ${DP} --datadir ${DATASETDIR}/ --resultdir ${RESULTDIR}/${MODELTYPE}${MODALITYTYPE}${EXPNAME} --modeltype ${MODELTYPE} --modalitytypes ${MODALITYTYPE} --perffile ${PERFFILE} --vitweights ${VITWEIGHTS}

Generated Dataset

EXPNAME=FAIRDIFFUSIONGLAUCOMA PERFFILE=${MODELTYPE}${MODALITYTYPE}${EXPNAME}.csv python scripts/trainglaucomafair.py --task cls --fairdiffusion --initialmodel stabilityai/stable-diffusion-2-1 --modelpath /checkpoint-/unet --epochs 50 --batchsize ${BATCHSIZE} --blr ${BLR} --minlr 1e-6 --warmupepochs 5 --weightdecay ${WD} --layerdecay ${LD} --droppath ${DP} --datadir ${DATASETDIR}/ --resultdir ${RESULTDIR}/${MODELTYPE}${MODALITYTYPE}${EXPNAME} --modeltype ${MODELTYPE} --modalitytypes ${MODALITYTYPE} --perffile ${PERFFILE} --vitweights ${VITWEIGHTS}

EXPNAME=FAIRDIFFUSIONMDSEVERITY PERFFILE=${MODELTYPE}${MODALITYTYPE}${EXPNAME}.csv python scripts/trainglaucomafair.py --task mdseverity --fairdiffusion --initialmodel stabilityai/stable-diffusion-2-1 --modelpath /checkpoint-/unet --epochs 50 --batchsize ${BATCHSIZE} --blr ${BLR} --minlr 1e-6 --warmupepochs 5 --weightdecay ${WD} --layerdecay ${LD} --droppath ${DP} --datadir ${DATASETDIR}/ --resultdir ${RESULTDIR}/${MODELTYPE}${MODALITYTYPE}${EXPNAME} --modeltype ${MODELTYPE} --modalitytypes ${MODALITYTYPE} --perffile ${PERFFILE} --vitweights ${VITWEIGHTS}

EXPNAME=FAIRDIFFUSIONCDRSTATUS PERFFILE=${MODELTYPE}${MODALITYTYPE}${EXPNAME}.csv python scripts/trainglaucomafair.py --task cdrstatus --fairdiffusion --initialmodel stabilityai/stable-diffusion-2-1 --modelpath /checkpoint-/unet --epochs 50 --batchsize ${BATCHSIZE} --blr ${BLR} --minlr 1e-6 --warmupepochs 5 --weightdecay ${WD} --layerdecay ${LD} --droppath ${DP} --datadir ${DATASETDIR}/ --resultdir ${RESULTDIR}/${MODELTYPE}${MODALITYTYPE}${EXPNAME} --modeltype ${MODELTYPE} --modalitytypes ${MODALITYTYPE} --perffile ${PERFFILE} --vitweights ${VITWEIGHTS}

EXPNAME=FAIRDIFFUSIONSESTATUS PERFFILE=${MODELTYPE}${MODALITYTYPE}${EXPNAME}.csv python scripts/trainglaucomafair.py --task sestatus --fairdiffusion --initialmodel stabilityai/stable-diffusion-2-1 --modelpath /checkpoint-/unet --epochs 50 --batchsize ${BATCHSIZE} --blr ${BLR} --minlr 1e-6 --warmupepochs 5 --weightdecay ${WD} --layerdecay ${LD} --droppath ${DP} --datadir ${DATASETDIR}/ --resultdir ${RESULTDIR}/${MODELTYPE}${MODALITYTYPE}${EXPNAME} --modeltype ${MODELTYPE} --modalitytypes ${MODALITYTYPE} --perffile ${PERFFILE} --vitweights ${VITWEIGHTS}

```

Evaluate Fairness of Diffusion Model (Classification) - EfficientNet ``` cd classification_codebase

DATASETDIR= RESULTDIR= MODELTYPE=( efficientnet ) NUMEPOCH=10 MODALITYTYPE='slofundus' ATTRIBUTE_TYPE=race

OPTIMIZER='adamw' OPTIMIZERARGUMENTS='{"lr": 0.001, "weightdecay": 0.01}'

SCHEDULER='steplr' SCHEDULERARGUMENTS='{"step_size": 30, "gamma": 0.1}'

LR=1e-3

Baselines

BATCHSIZE=6 EXPNAME=BASELINEGLAUCOMA PERFFILE=${MODELTYPE}${MODALITYTYPE}${EXPNAME}.csv python scripts/trainglaucomafair.py --task cls --datadir ${DATASETDIR}/ --resultdir ${RESULTDIR}/${MODELTYPE}${MODALITYTYPE}${EXPNAME} --modeltype ${MODELTYPE} --imagesize 200 --lr ${LR} --weight-decay 0. --momentum 0.1 --batchsize ${BATCHSIZE} --epochs ${NUMEPOCH} --modalitytypes ${MODALITYTYPE} --perffile ${PERFFILE}

BATCHSIZE=6 EXPNAME=BASELINEMDSEVERITY PERFFILE=${MODELTYPE}${MODALITYTYPE}${EXPNAME}.csv python scripts/trainglaucomafair.py --task mdseverity --datadir ${DATASETDIR}/ --resultdir ${RESULTDIR}/${MODELTYPE}${MODALITYTYPE}${EXPNAME} --modeltype ${MODELTYPE} --imagesize 200 --lr ${LR} --weight-decay 0. --momentum 0.1 --batchsize ${BATCHSIZE} --epochs ${NUMEPOCH} --modalitytypes ${MODALITYTYPE} --perffile ${PERFFILE}

BATCHSIZE=16 EXPNAME=BASELINECDRSTATUS PERFFILE=${MODELTYPE}${MODALITYTYPE}${EXPNAME}.csv python scripts/trainglaucomafair.py --task cdrstatus --datadir ${DATASETDIR}/ --resultdir ${RESULTDIR}/${MODELTYPE}${MODALITYTYPE}${EXPNAME} --modeltype ${MODELTYPE} --imagesize 200 --lr ${LR} --weight-decay 0. --momentum 0.1 --batchsize ${BATCHSIZE} --epochs ${NUMEPOCH} --modalitytypes ${MODALITYTYPE} --perffile ${PERFFILE}

BATCHSIZE=16 EXPNAME=BASELINESESTATUS PERFFILE=${MODELTYPE}${MODALITYTYPE}${EXPNAME}.csv python scripts/trainglaucomafair.py --task sestatus --datadir ${DATASETDIR}/ --resultdir ${RESULTDIR}/${MODELTYPE}${MODALITYTYPE}${EXPNAME} --modeltype ${MODELTYPE} --imagesize 200 --lr ${LR} --weight-decay 0. --momentum 0.1 --batchsize ${BATCHSIZE} --epochs ${NUMEPOCH} --modalitytypes ${MODALITYTYPE} --perffile ${PERFFILE}

Generated Dataset

BATCHSIZE=6 EXPNAME=FAIRDIFFUSIONGLAUCOMA PERFFILE=${MODELTYPE}${MODALITYTYPE}${EXPNAME}.csv python scripts/trainglaucomafair.py --task cls --fairdiffusion --initialmodel stabilityai/stable-diffusion-2-1 --modelpath <OUTPUTDIR>/checkpoint-/unet --datadir ${DATASETDIR}/ --resultdir ${RESULTDIR}/${MODELTYPE}${MODALITYTYPE}${EXPNAME} --modeltype ${MODELTYPE} --imagesize 200 --lr ${LR} --weight-decay 0. --momentum 0.1 --batchsize ${BATCHSIZE} --epochs ${NUMEPOCH} --modalitytypes ${MODALITYTYPE} --perffile ${PERF_FILE}

BATCHSIZE=6 EXPNAME=FAIRDIFFUSIONMDSEVERITY PERFFILE=${MODELTYPE}${MODALITYTYPE}${EXPNAME}.csv python scripts/trainglaucomafair.py --task mdseverity --fairdiffusion --initialmodel stabilityai/stable-diffusion-2-1 --modelpath <OUTPUTDIR>/checkpoint-/unet --datadir ${DATASETDIR}/ --resultdir ${RESULTDIR}/${MODELTYPE}${MODALITYTYPE}${EXPNAME} --modeltype ${MODELTYPE} --imagesize 200 --lr ${LR} --weight-decay 0. --momentum 0.1 --batchsize ${BATCHSIZE} --epochs ${NUMEPOCH} --modalitytypes ${MODALITYTYPE} --perffile ${PERF_FILE}

BATCHSIZE=16 EXPNAME=FAIRDIFFUSIONCDRSTATUS PERFFILE=${MODELTYPE}${MODALITYTYPE}${EXPNAME}.csv python scripts/trainglaucomafair.py --task cdrstatus --fairdiffusion --initialmodel stabilityai/stable-diffusion-2-1 --modelpath <OUTPUTDIR>/checkpoint-/unet --datadir ${DATASETDIR}/ --resultdir ${RESULTDIR}/${MODELTYPE}${MODALITYTYPE}${EXPNAME} --modeltype ${MODELTYPE} --imagesize 200 --lr ${LR} --weight-decay 0. --momentum 0.1 --batchsize ${BATCHSIZE} --epochs ${NUMEPOCH} --modalitytypes ${MODALITYTYPE} --perffile ${PERF_FILE}

BATCHSIZE=16 EXPNAME=FAIRDIFFUSIONSESTATUS PERFFILE=${MODELTYPE}${MODALITYTYPE}${EXPNAME}.csv python scripts/trainglaucomafair.py --task sestatus --fairdiffusion --initialmodel stabilityai/stable-diffusion-2-1 --modelpath <OUTPUTDIR>/checkpoint-/unet --datadir ${DATASETDIR}/ --resultdir ${RESULTDIR}/${MODELTYPE}${MODALITYTYPE}${EXPNAME} --modeltype ${MODELTYPE} --imagesize 200 --lr ${LR} --weight-decay 0. --momentum 0.1 --batchsize ${BATCHSIZE} --epochs ${NUMEPOCH} --modalitytypes ${MODALITYTYPE} --perffile ${PERF_FILE}

```

Citation

If you find this project interesting or useful for your research, please cite @article{FairDiffusion_Science_Advances_2025, author = {Yan Luo and Muhammad Osama Khan and Congcong Wen and Muhammad Muneeb Afzal and Titus Fidelis Wuermeling and Min Shi and Yu Tian and Yi Fang and Mengyu Wang }, title = {FairDiffusion: Enhancing equity in latent diffusion models via fair Bayesian perturbation}, journal = {Science Advances}, volume = {11}, number = {14}, pages = {eads4593}, year = {2025} }

Owner

Name: Harvard Ophthalmology AI Lab
Login: Harvard-Ophthalmology-AI-Lab
Kind: organization
Email: harvardophai@gmail.com
Location: United States of America

Twitter: HarvardOphAI
Repositories: 1
Profile: https://github.com/Harvard-Ophthalmology-AI-Lab

Engaging in research on AI in ophthalmology and vision science. Lab website: ophai.hms.harvard.edu

Citation (CITATION.cff)

cff-version: 1.2.0
title: 'Diffusers: State-of-the-art diffusion models'
message: >-
  If you use this software, please cite it using the
  metadata from this file.
type: software
authors:
  - given-names: Patrick
    family-names: von Platen
  - given-names: Suraj
    family-names: Patil
  - given-names: Anton
    family-names: Lozhkov
  - given-names: Pedro
    family-names: Cuenca
  - given-names: Nathan
    family-names: Lambert
  - given-names: Kashif
    family-names: Rasul
  - given-names: Mishig
    family-names: Davaadorj
  - given-names: Thomas
    family-names: Wolf
repository-code: 'https://github.com/huggingface/diffusers'
abstract: >-
  Diffusers provides pretrained diffusion models across
  multiple modalities, such as vision and audio, and serves
  as a modular toolbox for inference and training of
  diffusion models.
keywords:
  - deep-learning
  - pytorch
  - image-generation
  - hacktoberfest
  - diffusion
  - text2image
  - image2image
  - score-based-generative-modeling
  - stable-diffusion
  - stable-diffusion-diffusers
license: Apache-2.0
version: 0.12.1

GitHub Events

Total

Watch event: 8
Member event: 1
Push event: 4
Fork event: 4

Last Year

Watch event: 8
Member event: 1
Push event: 4
Fork event: 4

Dependencies

docker/diffusers-flax-cpu/Dockerfile docker

ubuntu 20.04 build

docker/diffusers-flax-tpu/Dockerfile docker

ubuntu 20.04 build

docker/diffusers-onnxruntime-cpu/Dockerfile docker

ubuntu 20.04 build

docker/diffusers-onnxruntime-cuda/Dockerfile docker

nvidia/cuda 11.6.2-cudnn8-devel-ubuntu20.04 build

docker/diffusers-pytorch-compile-cuda/Dockerfile docker

nvidia/cuda 12.1.0-runtime-ubuntu20.04 build

docker/diffusers-pytorch-cpu/Dockerfile docker

ubuntu 20.04 build

docker/diffusers-pytorch-cuda/Dockerfile docker

nvidia/cuda 12.1.0-runtime-ubuntu20.04 build

docker/diffusers-pytorch-xformers-cuda/Dockerfile docker

nvidia/cuda 12.1.0-runtime-ubuntu20.04 build

classification_codebase/requirements.txt pypi

blobfile >=1.3.3
fairlearn >=0.9.0
opencv-python *
pandas >=2.0.3
scikit-image >=0.19.3
scikit-learn >=1.1.2
torch >=2.1.0
torchvision >=0.15.2

examples/consistency_distillation/requirements.txt pypi

Jinja2 *
accelerate >=0.16.0
ftfy *
tensorboard *
torchvision *
transformers >=4.25.1
webdataset *

examples/controlnet/requirements.txt pypi

accelerate >=0.16.0
datasets *
ftfy *
tensorboard *
torchvision *
transformers >=4.25.1

examples/controlnet/requirements_flax.txt pypi

Jinja2 *
datasets *
flax *
ftfy *
optax *
tensorboard *
torch *
torchvision *
transformers >=4.25.1

examples/controlnet/requirements_sdxl.txt pypi

Jinja2 *
accelerate >=0.16.0
datasets *
ftfy *
tensorboard *
torchvision *
transformers >=4.25.1
wandb *

examples/custom_diffusion/requirements.txt pypi

Jinja2 *
accelerate *
ftfy *
tensorboard *
torchvision *
transformers >=4.25.1

examples/dreambooth/requirements.txt pypi

Jinja2 *
accelerate >=0.16.0
ftfy *
peft ==0.7.0
tensorboard *
torchvision *
transformers >=4.25.1

examples/dreambooth/requirements_flax.txt pypi

Jinja2 *
flax *
ftfy *
optax *
tensorboard *
torch *
torchvision *
transformers >=4.25.1

examples/dreambooth/requirements_sdxl.txt pypi

Jinja2 *
accelerate >=0.16.0
ftfy *
peft ==0.7.0
tensorboard *
torchvision *
transformers >=4.25.1

examples/instruct_pix2pix/requirements.txt pypi

accelerate >=0.16.0
datasets *
ftfy *
tensorboard *
torchvision *
transformers >=4.25.1

examples/kandinsky2_2/text_to_image/requirements.txt pypi

Jinja2 *
accelerate >=0.16.0
datasets *
ftfy *
tensorboard *
torchvision *
transformers >=4.25.1

examples/research_projects/colossalai/requirement.txt pypi

Jinja2 *
diffusers *
ftfy *
tensorboard *
torch *
torchvision *
transformers *

examples/research_projects/diffusion_dpo/requirements.txt pypi

Jinja2 *
accelerate >=0.16.0
ftfy *
peft *
tensorboard *
torchvision *
transformers >=4.25.1
wandb *

examples/research_projects/dreambooth_inpaint/requirements.txt pypi

Jinja2 *
accelerate >=0.16.0
diffusers ==0.9.0
ftfy *
tensorboard *
torchvision *
transformers >=4.21.0

examples/research_projects/intel_opts/textual_inversion/requirements.txt pypi

Jinja2 *
accelerate >=0.16.0
ftfy *
intel_extension_for_pytorch >=1.13
tensorboard *
torchvision *
transformers >=4.21.0

examples/research_projects/intel_opts/textual_inversion_dfq/requirements.txt pypi

accelerate *
ftfy *
modelcards *
neural-compressor *
tensorboard *
torchvision *
transformers >=4.25.0

examples/research_projects/lora/requirements.txt pypi

Jinja2 *
accelerate >=0.16.0
datasets *
ftfy *
tensorboard *
torchvision *
transformers >=4.25.1

examples/research_projects/multi_subject_dreambooth/requirements.txt pypi

Jinja2 *
accelerate >=0.16.0
ftfy *
tensorboard *
torchvision *
transformers >=4.25.1

examples/research_projects/multi_subject_dreambooth_inpainting/requirements.txt pypi

Jinja2 *
accelerate >=0.16.0
datasets >=2.16.0
ftfy *
tensorboard *
torchvision *
transformers >=4.25.1
wandb >=0.16.1

examples/research_projects/multi_token_textual_inversion/requirements.txt pypi

Jinja2 *
accelerate >=0.16.0
ftfy *
tensorboard *
torchvision *
transformers >=4.25.1

examples/research_projects/multi_token_textual_inversion/requirements_flax.txt pypi

Jinja2 *
flax *
ftfy *
optax *
tensorboard *
torch *
torchvision *
transformers >=4.25.1

examples/research_projects/onnxruntime/text_to_image/requirements.txt pypi

accelerate >=0.16.0
datasets *
ftfy *
modelcards *
tensorboard *
torchvision *
transformers >=4.25.1

examples/research_projects/onnxruntime/textual_inversion/requirements.txt pypi

accelerate >=0.16.0
ftfy *
modelcards *
tensorboard *
torchvision *
transformers >=4.25.1

examples/research_projects/onnxruntime/unconditional_image_generation/requirements.txt pypi

accelerate >=0.16.0
datasets *
tensorboard *
torchvision *

examples/research_projects/realfill/requirements.txt pypi

Jinja2 ==3.1.3
accelerate ==0.23.0
diffusers ==0.20.1
ftfy ==6.1.1
peft ==0.5.0
tensorboard ==2.14.0
torch ==2.0.1
torchvision >=0.16
transformers ==4.36.0

examples/t2i_adapter/requirements.txt pypi

accelerate >=0.16.0
datasets *
ftfy *
safetensors *
tensorboard *
torchvision *
transformers >=4.25.1
wandb *

examples/text_to_image/requirements.txt pypi

Jinja2 *
accelerate >=0.16.0
datasets *
ftfy *
peft ==0.7.0
tensorboard *
torchvision *
transformers >=4.25.1

examples/text_to_image/requirements_flax.txt pypi

Jinja2 *
datasets *
flax *
ftfy *
optax *
tensorboard *
torch *
torchvision *
transformers >=4.25.1

examples/text_to_image/requirements_sdxl.txt pypi

Jinja2 *
accelerate >=0.22.0
datasets *
ftfy *
peft ==0.7.0
tensorboard *
torchvision *
transformers >=4.25.1

examples/textual_inversion/requirements.txt pypi

Jinja2 *
accelerate >=0.16.0
ftfy *
tensorboard *
torchvision *
transformers >=4.25.1

examples/textual_inversion/requirements_flax.txt pypi

Jinja2 *
flax *
ftfy *
optax *
tensorboard *
torch *
torchvision *
transformers >=4.25.1

examples/unconditional_image_generation/requirements.txt pypi

accelerate >=0.16.0
datasets *
torchvision *

examples/wuerstchen/text_to_image/requirements.txt pypi

accelerate >=0.16.0
bitsandbytes *
deepspeed *
huggingface-cli *
peft >=0.6.0
torchvision *
transformers >=4.25.1
wandb *

pyproject.toml pypi

requirements.txt pypi

blobfile *
botorch *
datasets ==2.16.1
diffusers ==0.26.0
open_clip_torch *
scikit-learn ==1.4.0
scipy ==1.12.0
tensorboard ==2.13.0
timm ==0.4.12
torch *
torch-fidelity ==0.3.0
torchaudio *
torchmetrics ==1.3.0.post0
torchvision *
transformers ==4.36.2

setup.py pypi

deps *

environment.yml conda

_libgcc_mutex 0.1
_openmp_mutex 4.5
argon2-cffi 21.3.0
argon2-cffi-bindings 21.2.0
asttokens 2.0.5
attrs 21.4.0
backcall 0.2.0
beautifulsoup4 4.11.1
binutils_impl_linux-64 2.35.1
binutils_linux-64 2.35.1
blas 1.0
bleach 4.1.0
blosc 1.21.0
bottleneck 1.3.4
brotli 1.0.9
brotli-bin 1.0.9
brotlipy 0.7.0
brunsli 0.1
bzip2 1.0.8
ca-certificates 2023.05.30
certifi 2023.5.7
cffi 1.15.0
cfitsio 3.470
charls 2.2.0
charset-normalizer 2.0.4
cloudpickle 2.0.0
colorama 0.4.5
cryptography 36.0.0
cudatoolkit 11.3.1
cycler 0.11.0
cytoolz 0.11.0
dask-core 2022.5.0
dbus 1.13.6
debugpy 1.5.1
decorator 5.1.1
defusedxml 0.7.1
entrypoints 0.4
executing 0.8.3
expat 2.2.10
ffmpeg 4.3
fontconfig 2.13.1
fonttools 4.25.0
freetype 2.11.0
gcc_impl_linux-64 9.3.0
gcc_linux-64 9.3.0
gh 2.7.0
giflib 5.2.1
git 2.23.0
glib 2.69.1
gmp 6.2.1
gnutls 3.6.15
gst-plugins-base 1.14.0
gstreamer 1.14.0
gxx_impl_linux-64 9.3.0
gxx_linux-64 9.3.0
icu 58.2
idna 3.3
imagecodecs 2021.8.26
imageio 2.9.0
intel-openmp 2021.4.0
ipykernel 6.9.1
ipython 8.3.0
ipython_genutils 0.2.0
ipywidgets 7.6.5
jedi 0.18.1
jinja2 3.0.3
jpeg 9d
jsonschema 4.4.0
jupyter 1.0.0
jupyter_client 7.2.2
jupyter_console 6.4.3
jupyter_core 4.10.0
jupyterlab_pygments 0.1.2
jupyterlab_widgets 1.0.0
jxrlib 1.1
kernel-headers_linux-64 2.6.32
kiwisolver 1.3.2
krb5 1.18.2
lame 3.100
lcms2 2.12
ld_impl_linux-64 2.35.1
lerc 3.0
libaec 1.0.4
libbrotlicommon 1.0.9
libbrotlidec 1.0.9
libbrotlienc 1.0.9
libcurl 7.71.1
libdeflate 1.8
libedit 3.1.20191231
libffi 3.3
libgcc-devel_linux-64 9.3.0
libgcc-ng 9.3.0
libgfortran-ng 7.5.0
libgfortran4 7.5.0
libgomp 9.3.0
libiconv 1.15
libidn2 2.3.2
libpng 1.6.37
libsodium 1.0.18
libssh2 1.9.0
libstdcxx-devel_linux-64 9.3.0
libstdcxx-ng 9.3.0
libtasn1 4.16.0
libtiff 4.2.0
libunistring 0.9.10
libuuid 1.0.3
libuv 1.40.0
libwebp 1.2.2
libwebp-base 1.2.2
libxcb 1.13
libxml2 2.9.12
libzopfli 1.0.3
locket 1.0.0
lz4-c 1.9.3
markupsafe 2.1.1
matplotlib 3.5.1
matplotlib-base 3.5.1
matplotlib-inline 0.1.2
mistune 0.8.4
mkl 2021.4.0
mkl-service 2.4.0
mkl_fft 1.3.1
mkl_random 1.2.2
munkres 1.1.4
nbclient 0.5.13
nbconvert 6.4.4
nbformat 5.3.0
ncurses 6.3
nest-asyncio 1.5.5
nettle 3.7.3
networkx 2.7.1
notebook 6.4.11
numexpr 2.8.1
openh264 2.1.1
openjpeg 2.4.0
openssl 1.1.1u
pandocfilters 1.5.0
parso 0.8.3
partd 1.2.0
pcre 8.45
perl 5.26.2
pexpect 4.8.0
pickleshare 0.7.5
pillow 9.0.1
prometheus_client 0.13.1
prompt-toolkit 3.0.20
prompt_toolkit 3.0.20
pthread-stubs 0.4
ptyprocess 0.7.0
pure_eval 0.2.2
pycparser 2.21
pyopenssl 22.0.0
pyparsing 3.0.8
pyqt 5.9.2
pyrsistent 0.18.0
pysocks 1.7.1
python 3.9.12
python-dateutil 2.8.2
python-fastjsonschema 2.15.1
python_abi 3.9
pytorch-mutex 1.0
pywavelets 1.3.0
pyyaml 6.0
pyzmq 22.3.0
qt 5.9.7
qtconsole 5.3.0
qtpy 2.0.1
readline 8.1.2
scikit-image 0.19.2
send2trash 1.8.0
sip 4.19.13
six 1.16.0
snappy 1.1.9
soupsieve 2.3.1
sqlite 3.38.2
stack_data 0.2.0
sysroot_linux-64 2.12
terminado 0.13.1
testpath 0.6.0
threadpoolctl 2.2.0
tifffile 2021.7.2
tk 8.6.11
toolz 0.11.2
tornado 6.1
traitlets 5.1.1
typing-extensions 4.1.1
typing_extensions 4.1.1
urllib3 1.26.8
wcwidth 0.2.5
webencodings 0.5.1
wheel 0.37.1
widgetsnbextension 3.5.2
xorg-libxau 1.0.9
xorg-libxdmcp 1.1.3
xz 5.2.5
yaml 0.2.5
zeromq 4.3.4
zfp 0.5.5
zlib 1.2.11
zstd 1.4.9

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Open Source Science

fairdiffusion

Science Score: 44.0%

Repository

Basic Info

Statistics

Metadata Files

README.md

FairDiffusion

Requirements

FairGenMed Dataset

Experiments

Baselines

Generated Dataset

Baselines

Generated Dataset

Citation

Owner

Citation (CITATION.cff)

GitHub Events

Total

Last Year

Dependencies