🤗 Diffusers is the go-to library for state-of-the-art pretrained diffusion models for generating images, audio, and even 3D structures of molecules. Whether you're looking for a simple inference solution or training your own diffusion models, 🤗 Diffusers is a modular toolbox that supports both. Our library is designed with a focus on usability over performance, simple over easy, and customizability over abstractions.

🤗 Diffusers offers three core components:

• State-of-the-art diffusion pipelines that can be run in inference with just a few lines of code.
• Interchangeable noise schedulers for different diffusion speeds and output quality.
• Pretrained models that can be used as building blocks, and combined with schedulers, for creating your own end-to-end diffusion systems.

Installation

We recommend installing 🤗 Diffusers in a virtual environment from PyPI or Conda. For more details about installing PyTorch and Flax, please refer to their official documentation.

PyTorch

With pip (official package):

bash pip install --upgrade diffusers[torch]

With conda (maintained by the community):

sh conda install -c conda-forge diffusers

Flax

With pip (official package):

bash pip install --upgrade diffusers[flax]

Apple Silicon (M1/M2) support

Please refer to the How to use Stable Diffusion in Apple Silicon guide.

Quickstart

Generating outputs is super easy with 🤗 Diffusers. To generate an image from text, use the from_pretrained method to load any pretrained diffusion model (browse the Hub for 30,000+ checkpoints):

```python from diffusers import DiffusionPipeline import torch

pipeline = DiffusionPipeline.frompretrained("stable-diffusion-v1-5/stable-diffusion-v1-5", torchdtype=torch.float16) pipeline.to("cuda") pipeline("An image of a squirrel in Picasso style").images[0] ```

You can also dig into the models and schedulers toolbox to build your own diffusion system:

```python from diffusers import DDPMScheduler, UNet2DModel from PIL import Image import torch

scheduler = DDPMScheduler.frompretrained("google/ddpm-cat-256") model = UNet2DModel.frompretrained("google/ddpm-cat-256").to("cuda") scheduler.set_timesteps(50)

samplesize = model.config.samplesize noise = torch.randn((1, 3, samplesize, samplesize), device="cuda") input = noise

for t in scheduler.timesteps: with torch.nograd(): noisyresidual = model(input, t).sample prevnoisysample = scheduler.step(noisyresidual, t, input).prevsample input = prevnoisysample

image = (input / 2 + 0.5).clamp(0, 1) image = image.cpu().permute(0, 2, 3, 1).numpy()[0] image = Image.fromarray((image * 255).round().astype("uint8")) image ```

Check out the Quickstart to launch your diffusion journey today!

How to navigate the documentation

| Documentation | What can I learn? | |---------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Tutorial | A basic crash course for learning how to use the library's most important features like using models and schedulers to build your own diffusion system, and training your own diffusion model. | | Loading | Guides for how to load and configure all the components (pipelines, models, and schedulers) of the library, as well as how to use different schedulers. | | Pipelines for inference | Guides for how to use pipelines for different inference tasks, batched generation, controlling generated outputs and randomness, and how to contribute a pipeline to the library. | | Optimization | Guides for how to optimize your diffusion model to run faster and consume less memory. | | Training | Guides for how to train a diffusion model for different tasks with different training techniques. |

Popular Tasks & Pipelines