Reward — a score for an outcome
A number saying how good an outcome was. For a chat response it comes from a learned reward model; for maths or code it can come from a checker.
Section 21.4 trains a reward model from pairwise human preferences; section 22.3 uses verifiable rewards.
Where the course uses it
Related terms
- Policy — a rule for choosing actionsIn reinforcement learning, a probability distribution over actions given the current state. A language model is a policy over next tokens.
- Advantage — reward relative to a baselineHow much better a response scored than a typical response, R − b. Policy gradients weighted by the advantage are far less noisy than those weighted by the raw reward.