Chapter 20 · Hands-on: more projects
Make images on your computer
Open image models can run locally if your computer has enough memory, often with a GPU.
Chapter 20 · Hands-on: more projects
Open image models can run locally if your computer has enough memory, often with a GPU.
Your sentence goes in…
A script that loads an image model into your computer’s memory and turns a sentence into a picture. The model starts from random noise and cleans it up step by step, as in the diffusion lesson.
Image models are much bigger than the speech and text-embedding models in the earlier tutorials. This one is honest about that: it needs a lot of memory, and without a graphics card it is slow. Use the fit helper first and decide whether to follow along, or just read to see how it works.
You need Python as in Set up Python and a virtual environment. Windows, macOS and Linux desktops are supported. This tutorial uses diffusers, one popular open-source library for image models.
Scroll to the helper under this tutorial and enter your hardware. If it says Probably too much, you can still read on and follow later on a stronger computer. If it says Possible, but slow, use a small model, a small picture size and be patient.
Check: You know which kind of model your computer can try.
As in the Python setup lesson, create a project folder and a private box for its packages.
mkdir image-lab
cd image-lab
python -m venv .venv
.venv\Scripts\Activate.ps1 Check: Your prompt starts with (.venv).
PyTorch does the heavy maths. diffusers is the image-model library. transformers reads your text prompt, and accelerate and safetensors help load big model files.
pip install torch diffusers transformers accelerate safetensorsCheck: The install finishes without red errors. It may take several minutes.
Create make_image.py and paste this in. The first block picks the best hardware it finds: a graphics card (cuda), an Apple chip (mps) or the plain processor (cpu). float16 stores the model’s numbers in half the space, which saves memory on graphics hardware.
The seed starts the random noise. The same seed and prompt give the same picture. Steps is how many clean-up rounds the model does.
import sys
import torch
from diffusers import AutoPipelineForText2Image
MODEL_ID = "stable-diffusion-v1-5/stable-diffusion-v1-5" # one example; use any text-to-image model that fits your memory
STEPS = 25 # more steps: slower, sometimes cleaner
SIZE = 512 # width and height in pixels; smaller uses less memory
SEED = 42 # same seed + same prompt = same picture
prompt = " ".join(sys.argv[1:]) or "a watercolour painting of a lighthouse at sunrise"
if torch.cuda.is_available():
device, dtype = "cuda", torch.float16
elif torch.backends.mps.is_available():
device, dtype = "mps", torch.float16
else:
device, dtype = "cpu", torch.float32
print("Using:", device)
pipe = AutoPipelineForText2Image.from_pretrained(MODEL_ID, torch_dtype=dtype)
pipe = pipe.to(device)
pipe.enable_attention_slicing() # trades a little speed for lower memory use
generator = torch.Generator("cpu").manual_seed(SEED)
image = pipe(prompt, num_inference_steps=STEPS, height=SIZE, width=SIZE, generator=generator).images[0]
name = f"picture-{SEED}.png"
image.save(name)
print("Saved", name) Check: The file is saved as make_image.py.
Run the script with your sentence after its name. Keep the first prompt simple.
python make_image.py "a watercolour painting of a lighthouse at sunrise" Check: You see Using: …, a progress bar, then Saved picture-42.png, and the picture opens from the folder.
Change SEED to get a different picture of the same prompt. Change STEPS to see whether more clean-up rounds help. Lower SIZE (for example 384) to save memory. Describe style, subject and light in your prompt, for example “a paper-cut tiger in a green forest, soft morning light”.
Check: You can say what each of the three settings does to the picture or the time taken.
Pick your hardware and memory. This is a rough guide that runs nothing on your computer.
Looks goodShould fit on your graphics card with room to spare. Pictures take seconds to a minute.
A rough planning guide with round numbers, not a measurement. Find your graphics memory in your system’s settings or task manager. Other brands of graphics card may work with extra setup; check the library’s documentation.
What do you see?
What does the seed control?
You have no graphics card. What should you expect?
Why check the model’s licence page?