Chapter 4 · The deep learning era (2006–2016)
Transfer learning
Reuse a model trained on a big task as the starting point for your small task.
Chapter 4 · The deep learning era (2006–2016)
Reuse a model trained on a big task as the starting point for your small task.
A model was trained on a huge task. Its inner layers learned useful features.
Most of the learning is already done. Only a small new part has to learn your task.
A model that starts with random numbers needs a lot of labelled examples. For many problems you only have a few.
The puzzle: tell the violet inside points from the coral ring. Slide the number of labelled examples, then switch the start.
Score = share of 240 unseen points classified correctly (50% is guessing). A simplified teaching version: tiny models, hand-made pretrained features, three numbers per point.
In transfer learning, what usually gets trained on your small data?
When does a pretrained start help the most?
In the lab, what happened to the from-scratch model as examples grew?