πΌ β expected value
Read "the expected value of X". It is the average value a random quantity takes if you draw it many, many times.
is what averages out to over a very large number of draws. Two rules are used in the course:
- It is linear, always: .
- For independent quantities (knowing one tells you nothing about the other), the expected value of a product splits: .
The second rule only holds under independence, and the proof behind the scaling depends on it.
Example
No single roll is 3.5, but the average over many rolls approaches it.
Where the course uses it
- 3.2 Expectation and Variance Β· Module 3, Probability and Informationfirst use
- 6.3 Why Divide by the Square Root of the Key Width Β· Module 6, Attention
- 21.1 A Reinforcement Learning Primer Β· Module 21, Preference Learning: RL, RLHF and DPO
- 21.2 Policy Gradients Β· Module 21, Preference Learning: RL, RLHF and DPO
- 21.3 PPO Β· Module 21, Preference Learning: RL, RLHF and DPO
- 21.5 RLHF End to End Β· Module 21, Preference Learning: RL, RLHF and DPO
Related terms
- Var β varianceRead "the variance of X". It measures how spread out a random quantity is around its average. Its square root is the standard deviation.
- The 1/βd_k scalingRead "one over root d k". Attention divides its scores by the square root of the head width so they keep a spread of about 1, however wide the head is.