Chapter 6 · The Transformer
Encoders, decoders and both
Encoders understand (BERT), decoders generate (GPT), and encoder-decoders translate (T5).
Chapter 6 · The Transformer
Encoders understand (BERT), decoders generate (GPT), and encoder-decoders translate (T5).
Encoder only: Reads the whole text, both ways. Makes: a meaning: a label, a score or a vector.
Same building blocks, arranged three ways. What goes in and what comes out tells you what each one is for.
An encoder reads the whole text and every token can look both left and right. It ends with a meaning: a label, a score or a vector. BERT-style models are encoders.
Six tasks. For each one, pick the shape that was built for it.
These answers show what each shape was designed for. The lines blur in practice: large decoder-only models are used for many of these tasks too.
Which shape reads the whole text in both directions?
Translating a sentence into another language best fits…
Which shape is a GPT-style model?