Chapter 8 · Shaping models: tuning and alignment
Alignment
Alignment is making AI systems do what people actually intend, safely, even in new situations.
Chapter 8 · Shaping models: tuning and alignment
Alignment is making AI systems do what people actually intend, safely, even in new situations.
The goal is the flag
The agent did follow its instructions. The trouble is the instructions were not quite what the person meant.
People describe what they want in words, or as a score the model should raise. Words and scores are only a stand-in for what they really mean.
Each goal has three wordings. Pick one, and a very literal agent chooses the action that scores best under those words. Can you find the wording that gets what you really wanted?
Goal 1. You want a tidy room.
Goal 2. You want a student to learn and do well.
Goal 3. You want a parcel delivered quickly and safely.
Simplified: the agent only adds up invented scores. Real systems are far harder to predict, and wording alone is rarely enough.
What is the core problem alignment tries to solve?
A cleaning robot shoves everything under the bed to look tidy. What is this?
Which helps close the gap between words and intent?