\infty

∞ — infinity

Read "infinity". It is not an ordinary number. It means "grows without limit", and in an attention mask, −∞ means "never allowed".

The attention mask uses $-\infty$ because adding it to any score makes the result $-\infty$ , and $e^{-\infty} = 0$ : the position gets zero weight. Code uses a large finite negative number instead, because true infinities cause NaN ("not a number") in some arithmetic, such as $\infty - \infty$ or $0 \times \infty$ . For example, a row where every position is masked with $-\infty$ would give NaN.

Example

e^{\,3 + (-\infty)} = e^{-\infty} = 0

Whatever the original score, a masked position ends with weight zero.

Where the course uses it

4.3 Logits, Softmax, and Cross-Entropy · Module 4, How a Model Learns From Datafirst use
5.4 Vanishing and Exploding Gradients · Module 5, Sequences Before Transformers
6.2 Scaled Dot-Product Attention, by Hand · Module 6, Attention
6.4 Masks · Module 6, Attention
8.4 Decoder-Only: Predicting the Next Token · Module 8, Three Families: Encoder, Encoder-Decoder, Decoder
12.3 Decoding: From Logits to Text · Module 12, Generation

Related terms

M — the attention maskA matrix of 0s and −∞ (minus infinity) added to the attention scores before softmax. A 0 leaves a position allowed; −∞ blocks it.
The exponential functionRead "e to the z" (also written exp(z)). It raises e ≈ 2.718 to the power z. The result is always positive: e⁰ = 1, and it shrinks toward 0 as z becomes very negative.
lim — limitRead "the limit as T goes to 0". It is the value a quantity gets closer and closer to as something is pushed toward an extreme, even if that extreme can't be plugged in directly.