x_i

Subscripts — picking one item

Read x_i as "x sub i" or "x i". A small lowered letter picks out one item: the i-th entry of a vector, or the i-th vector in a sequence.

Which of the two depends on the letter's font. $x_i$ (plain) is one number inside a vector; $\mathbf{x}_i$ (bold) is the whole vector for token $i$ . Two subscripts pick an entry of a matrix: $M_{ij}$ is the entry in row $i$ , column $j$ .

By habit, $i$ and $j$ count features or positions, $t$ counts time steps, $l$ counts layers, and $m$ and $n$ are positions in RoPE. Some subscripts are names rather than counters, such as the "model" in $d_{\text{model}}$ .

Example

M = \begin{bmatrix} 5 & 7 \\ 2 & 9 \end{bmatrix} \;\Longrightarrow\; M_{12} = 7

Row 1, column 2.

Where the course uses it

0.3 What You Need, and How the Course Is Organized · Module 0, Start Here: What an LLM Doesfirst use
1.2 Matrix Multiplication, Done by Hand · Module 1, Shapes and Linear Algebra
1.3 The Matrix Toolkit · Module 1, Shapes and Linear Algebra
1.4 Dot Products, Length, and Angle · Module 1, Shapes and Linear Algebra
1.5 Embeddings: From a Token to a Vector · Module 1, Shapes and Linear Algebra
1.6 Linear Projections: One Vector Into Three · Module 1, Shapes and Linear Algebra
2.2 Partial Derivatives and the Gradient · Module 2, Calculus and Optimization
2.3 The Chain Rule and the Jacobian · Module 2, Calculus and Optimization
2.5 Backpropagation · Module 2, Calculus and Optimization
3.4 Maximum Likelihood · Module 3, Probability and Information
3.6 The Sigmoid and Preferences Between Pairs · Module 3, Probability and Information
4.3 Logits, Softmax, and Cross-Entropy · Module 4, How a Model Learns From Data
5.6 The First Attention · Module 5, Sequences Before Transformers
6.3 Why Divide by the Square Root of the Key Width · Module 6, Attention
6.5 Attention Cannot See Word Order · Module 6, Attention
6.6 Multi-Head Attention · Module 6, Attention
7.1 Positional Encodings · Module 7, The Block and the Full Model
7.3 LayerNorm · Module 7, The Block and the Full Model
7.4 The Feed-Forward Network · Module 7, The Block and the Full Model
8.2 Encoder-Only: BERT and Masked Language Modeling · Module 8, Three Families: Encoder, Encoder-Decoder, Decoder
8.4 Decoder-Only: Predicting the Next Token · Module 8, Three Families: Encoder, Encoder-Decoder, Decoder
11.4 Tensor Parallelism · Module 11, Training Across Many GPUs
12.3 Decoding: From Logits to Text · Module 12, Generation
15.1 Rotary Position Embedding (RoPE) · Module 15, The LLaMA Block
15.2 Stretching RoPE, and ALiBi · Module 15, The LLaMA Block
15.3 RMSNorm · Module 15, The LLaMA Block
16.2 FlashAttention and the Online Softmax · Module 16, Efficient and Long Attention
17.2 Many Small Experts · Module 17, Mixture of Experts
17.4 Router Collapse and the Load-Balancing Loss · Module 17, Mixture of Experts
20.2 The Singular Value Decomposition · Module 20, Adaptation: Fine-Tuning, LoRA, Distillation
22.3 RL With Verifiable Rewards and GRPO · Module 22, Reasoning Models
24.4 Post-Training Quantization: AWQ and GPTQ · Module 24, Quantization and Number Formats
25.3 Contrastive Learning: CLIP and SigLIP · Module 25, Vision Transformers and Contrastive Learning

Related terms

Bold letters are vectorsA bold lowercase letter, like x in bold, is a vector (a list of numbers). A plain italic letter is one number. A capital letter is a matrix.
Superscript in parentheses — a labelRead x^(t) as "x at step t". A raised number or letter in parentheses is a label, such as a time step. It is never a power.