\beta

β — a shift, a smoothing factor or a strength, depending on the section

Read "beta". LayerNorm's learned shift (4.2), Swish's slope (4.3), Adam's averaging factors β1 and β2 (3.6), the drift penalty in RLHF and DPO (3.7), and the write strength of the delta rule (5.4).

These are unrelated uses that follow each paper's own notation. With a subscript number, $\beta_1$ and $\beta_2$ are Adam's; bold $\boldsymbol{\beta}$ is LayerNorm's shift vector; with a time subscript, $\beta_t$ is the delta rule's write strength. In RLHF and DPO, $\beta$ sets how far the trained model may move from the reference model.

Where the course uses it

7.3 LayerNorm · Module 7, The Block and the Full Modelfirst use
10.2 AdamW · Module 10, Pretraining
10.5 Scaling Laws: How Big, and on How Much Text · Module 10, Pretraining
15.3 RMSNorm · Module 15, The LLaMA Block
15.4 SwiGLU · Module 15, The LLaMA Block
18.2 The Gated Delta Rule · Module 18, Beyond Attention: Linear Attention and Hybrids
19.2 A Config File, Field by Field · Module 19, Reading a Model Card
21.5 RLHF End to End · Module 21, Preference Learning: RL, RLHF and DPO
21.6 Direct Preference Optimization · Module 21, Preference Learning: RL, RLHF and DPO

Related terms

α — a scale or weight whose meaning depends on the sectionRead "alpha". The course reuses it, each time as in the original paper: the LoRA scale (1.3), the load-balancing weight (5.3), the SmoothQuant exponent (4.7), and the CTC forward table (7.2).