Cases — an if/else
A large left brace with several lines is an if/else written vertically. Read each line as "this value, if this condition holds" and use the line whose condition is true.
Each row has a value, then a condition. Exactly one condition should be true for any input. It is if / elif / else in maths notation. The causal mask, the repetition penalty and the multimodal label mask are written this way.
Example
Absolute value: for x = -3 the second line applies, giving 3.
Where the course uses it
- 4.3 Logits, Softmax, and Cross-Entropy · Module 4, How a Model Learns From Datafirst use
- 8.4 Decoder-Only: Predicting the Next Token · Module 8, Three Families: Encoder, Encoder-Decoder, Decoder
- 12.3 Decoding: From Logits to Text · Module 12, Generation
- 26.4 Training on Images and Text Together · Module 26, Vision-Language Models
Related terms
- δᵢⱼ — Kronecker deltaRead "delta i j". It equals 1 when i and j are the same number and 0 when they differ. It is an if-statement written as a number.
- M — the attention maskA matrix of 0s and −∞ (minus infinity) added to the attention scores before softmax. A 0 leaves a position allowed; −∞ blocks it.