diffused

🤗 Generate images with diffusion models:

diffused <model> <prompt>

Quick Start

Text-to-image:

pipx run diffused segmind/tiny-sd "red apple"

Image-to-image:

pipx run diffused OFA-Sys/small-stable-diffusion-v0 "cat wizard" --image=https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/cat.png

Inpainting:

pipx run diffused kandinsky-community/kandinsky-2-2-decoder-inpaint "black cat" --image=https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/inpaint.png --mask-image=https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/inpaint_mask.png

Prerequisites

Python
pipx

CLI

Install the CLI:

pipx install diffused

`model`

Required (str): The diffusion model.

diffused segmind/SSD-1B "An astronaut riding a green horse"

See segmind/SSD-1B.

`prompt`

Required (str): The text prompt.

diffused dreamlike-art/dreamlike-photoreal-2.0 "cinematic photo of Godzilla eating sushi with a cat in a izakaya, 35mm photograph, film, professional, 4k, highly detailed"

`--negative-prompt`

Optional (str): What to exclude from the output image.

diffused stabilityai/stable-diffusion-2 "photo of an apple" --negative-prompt="blurry, bright photo, red"

With the short option:

diffused stabilityai/stable-diffusion-2 "photo of an apple" -np="blurry, bright photo, red"

`--image`

Optional (str): The input image path or URL. The initial image is used as a starting point for an image-to-image diffusion process.

diffused stabilityai/stable-diffusion-xl-refiner-1.0 "astronaut in a desert" --image=https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/img2img-init.png

With the short option:

diffused stabilityai/stable-diffusion-xl-refiner-1.0 "astronaut in a desert" -i=./local/image.png

`--mask-image`

Optional (str): The mask image path or URL. Inpainting replaces or edits specific areas of an image. Create a mask image to inpaint images.

diffused kandinsky-community/kandinsky-2-2-decoder-inpaint "black cat" --image=https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/inpaint.png --mask-image=https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/inpaint_mask.png

With the short option:

diffused kandinsky-community/kandinsky-2-2-decoder-inpaint "black cat" -i=inpaint.png -mi=inpaint_mask.png

`--output`

Optional (str): The output image filename.

diffused dreamlike-art/dreamlike-photoreal-2.0 "cat eating sushi" --output=cat.jpg

With the short option:

diffused dreamlike-art/dreamlike-photoreal-2.0 "cat eating sushi" -o=cat.jpg

`--width`

Optional (int): The output image width in pixels.

diffused stabilityai/stable-diffusion-xl-base-1.0 "dog in space" --width=1024

With the short option:

diffused stabilityai/stable-diffusion-xl-base-1.0 "dog in space" -W=1024

`--height`

Optional (int): The output image height in pixels.

diffused stabilityai/stable-diffusion-xl-base-1.0 "dog in space" --height=1024

With the short option:

diffused stabilityai/stable-diffusion-xl-base-1.0 "dog in space" -H=1024

`--number`

Optional (int): The number of output images. Defaults to 1.

diffused segmind/tiny-sd apple --number=2

With the short option:

diffused segmind/tiny-sd apple -n=2

`--guidance-scale`

Optional (int): How much the prompt influences the output image. A lower value leads to more deviation and creativity, whereas a higher value follows the prompt to a tee.

diffused stable-diffusion-v1-5/stable-diffusion-v1-5 "astronaut in a jungle" --guidance-scale=7.5

With the short option:

diffused stable-diffusion-v1-5/stable-diffusion-v1-5 "astronaut in a jungle" -gs=7.5

`--inference-steps`

Optional (int): The number of diffusion steps used during image generation. The more steps you use, the higher the quality, but the generation time will increase.

diffused CompVis/stable-diffusion-v1-4 "astronaut rides horse" --inference-steps=50

With the short option:

diffused CompVis/stable-diffusion-v1-4 "astronaut rides horse" -is=50

`--strength`

Optional (float): The noise added to the input image, which determines how much the output image deviates from the original image. Strength is used for image-to-image and inpainting tasks and is a multiplier to the number of denoising steps (--inference-steps).

diffused stabilityai/stable-diffusion-xl-refiner-1.0 "astronaut in swamp" --image=https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/img2img-sdxl-init.png --strength=0.5

With the short option:

diffused stabilityai/stable-diffusion-xl-refiner-1.0 "astronaut in swamp" -i=image.png -s=0.5

`--seed`

Optional (int): The seed for generating random numbers, ensuring reproducibility in image generation pipelines.

diffused stable-diffusion-v1-5/stable-diffusion-v1-5 "Labrador in the style of Vermeer" --seed=0

With the short option:

diffused stable-diffusion-v1-5/stable-diffusion-v1-5 "Labrador in the style of Vermeer" -S=1337

`--device`

Optional (str): The device to accelerate the computation (cpu, cuda, mps, xpu, xla, or meta).

diffused stable-diffusion-v1-5/stable-diffusion-v1-5 "astronaut on earth, 8k" --device=cuda

With the short option:

diffused stable-diffusion-v1-5/stable-diffusion-v1-5 "astronaut on earth, 8k" -d=cuda

`--no-safetensors`

Optional (bool): Whether to disable safetensors.

diffused runwayml/stable-diffusion-v1-5 "astronaut on mars" --no-safetensors

`--version`

Show the program's version number and exit:

diffused --version # diffused -v

`--help`

Show the help message and exit:

diffused --help # diffused -h

Script

Create a virtual environment:

python3 -m venv .venv

Activate the virtual environment:

source .venv/bin/activate

Install the package:

pip install diffused

Generate an image with a model and a prompt:

# script.py
from diffused import generate

images = generate(model="segmind/tiny-sd", prompt="apple")
images[0].save("apple.png")

Run the script:

python script.py

Open the image:

open apple.png

See the API documentation.

License

MIT

View Source

1"""
2.. include:: ../../README.md
3"""
4
5from .generate import generate
6
7__all__ = ["generate"]
8
9__version__ = "1.0.0"

def generate(**kwargs: Unpack[diffused.generate.Generate]) -> list[PIL.Image.Image]: View Source

26def generate(**kwargs: Unpack[Generate]) -> list[Image.Image]:
27    """
28    Generate image with diffusion model.
29
30    Args:
31        model (str): Diffusion model.
32        prompt (str): Text prompt.
33        negative_prompt (str): What to exclude from the generated image.
34        image (str): Input image path or URL.
35        mask_image (str): Mask image path or URL.
36        width (int): Generated image width in pixels.
37        height (int): Generated image height in pixels.
38        num_images_per_prompt (int): Number of images per prompt.
39        guidance_scale (float): How much the prompt influences image generation.
40        num_inference_steps (int): Number of diffusion steps used for generation.
41        strength (float): How much noise is added to the input image.
42        seed (int): Seed for generating reproducible images.
43        device (str): Device to accelerate computation (cpu, cuda, mps).
44        use_safetensors (bool): Whether to load safetensors.
45
46    Returns:
47        images (list[PIL.Image.Image]): Pillow images.
48    """
49    pipeline_args = {
50        "prompt": kwargs.get("prompt"),
51        "negative_prompt": kwargs.get("negative_prompt"),
52        "width": kwargs.get("width"),
53        "height": kwargs.get("height"),
54        "num_images_per_prompt": kwargs.get("num_images_per_prompt", 1),
55        "use_safetensors": kwargs.get("use_safetensors", True),
56    }
57
58    guidance_scale = kwargs.get("guidance_scale")
59    if guidance_scale is not None and guidance_scale >= 0:
60        pipeline_args["guidance_scale"] = guidance_scale
61
62    num_inference_steps = kwargs.get("num_inference_steps")
63    if num_inference_steps is not None and num_inference_steps >= 0:
64        pipeline_args["num_inference_steps"] = num_inference_steps
65
66    strength = kwargs.get("strength")
67    if strength is not None and strength >= 0:
68        pipeline_args["strength"] = strength
69
70    Pipeline = diffusers.AutoPipelineForText2Image
71
72    if kwargs.get("image"):
73        pipeline_args["image"] = diffusers.utils.load_image(kwargs.get("image"))
74        Pipeline = diffusers.AutoPipelineForImage2Image
75
76    if kwargs.get("mask_image"):
77        pipeline_args["mask_image"] = diffusers.utils.load_image(
78            kwargs.get("mask_image")
79        )
80        Pipeline = diffusers.AutoPipelineForInpainting
81
82    pipeline = Pipeline.from_pretrained(kwargs.get("model"))
83
84    device = kwargs.get("device")
85    if device:
86        pipeline.to(device)
87
88    seed = kwargs.get("seed")
89    if isinstance(seed, int):
90        pipeline_args["generator"] = torch.Generator(device=device).manual_seed(seed)
91
92    images = pipeline(**pipeline_args).images
93    return images

Generate image with diffusion model.

Args: model (str): Diffusion model. prompt (str): Text prompt. negative_prompt (str): What to exclude from the generated image. image (str): Input image path or URL. mask_image (str): Mask image path or URL. width (int): Generated image width in pixels. height (int): Generated image height in pixels. num_images_per_prompt (int): Number of images per prompt. guidance_scale (float): How much the prompt influences image generation. num_inference_steps (int): Number of diffusion steps used for generation. strength (float): How much noise is added to the input image. seed (int): Seed for generating reproducible images. device (str): Device to accelerate computation (cpu, cuda, mps). use_safetensors (bool): Whether to load safetensors.

Returns: images (list[PIL.Image.Image]): Pillow images.