Chapter 4 · The deep learning era (2006–2016)
Autoencoders
An autoencoder squeezes data through a bottleneck and rebuilds it, learning a compact code.
Chapter 4 · The deep learning era (2006–2016)
An autoencoder squeezes data through a bottleneck and rebuilds it, learning a compact code.
A picture goes in: 36 numbers
The encoder turns the picture into a few numbers, the code. The decoder tries to rebuild the picture from only that code. Whatever the code cannot hold is lost.
An autoencoder has an encoder that shrinks the input, and a decoder that expands it back. In the middle is the bottleneck: a layer with far fewer numbers than the input.
Pick one of the twelve shapes the autoencoder learned from, or draw your own. Then change the bottleneck size. A shape it has seen rebuilds sharply with a few numbers; your own scribble does not.
A teaching version: a linear autoencoder worked out exactly for these twelve shapes, not trained step by step. Real autoencoders use non-linear layers and thousands of examples. Pictures unlike the training set rebuild badly: the code is only good for what it learned.
What is an autoencoder trained to do?
You make the bottleneck narrower. The rebuilt picture will usually…
Why does a scribble unlike the training shapes rebuild badly?