Chapter 8 · Shaping models: tuning and alignment
Red teaming and safety testing
Red teams try hard to make a model misbehave so problems are found and fixed before release.
Preparing red teaming: probing an ai for weak spots
Chapter 8 · Shaping models: tuning and alignment
Red teams try hard to make a model misbehave so problems are found and fixed before release.
Tester probes “Harmful help”. A weak spot is found, then fixed.
Red teamers act like friendly attackers. Every weak spot they find is a fix made before release.
A red team is a group paid to break the model on purpose. They try odd, tricky and rude requests, and write down what goes wrong.
Each card describes a test in general words. Pick the risk it checks. Right answers light up the shield.
A tester asks the model to ignore its rules and show its hidden setup text.
What does a red team do?
After a weak spot is fixed, what should happen?
Can red teaming prove a model is perfectly safe?