Chapter 14 · Images, sound and video2020
Diffusion models
Diffusion models learn to remove noise step by step, turning static into an image.
Chapter 14 · Images, sound and video2020
Diffusion models learn to remove noise step by step, turning static into an image.
Noise left: 100%. Pure static. Nothing to see.
The model never draws the heart in one go. It only ever answers one small question: what would this look like with a little less noise?
Take a real picture and add noise, a little at a time, until only static is left. This forward process needs no learning. It is just adding random numbers.
Clear noise with a toy denoiser that knows three shapes. Try one step, then thirty. Press New random noise to see a different shape appear.
This is pure random noise. The denoiser's guess is a blur of all three shapes.
A teaching toy: the denoiser here is the exact best guess for just three shapes. A real model is a large neural network trained on millions of pictures.
What does a diffusion model start from when it makes a picture?
Why is training possible without anyone labelling anything?
In the toy, what happened with just one step?