\in

∈ — is in

Read "is in" or "belongs to". The thing on the left is a member of the set on the right.

$a \in S$ says that $a$ is one of the things in the set $S$ . In this course it mostly does one job: stating a shape. $\mathbf{q} \in \mathbb{R}^{64}$ means " $\mathbf{q}$ is a vector of 64 real numbers", like a type annotation in code.

It is also used with a written-out set: $i \in \{0, 1, \dots, |V|-1\}$ says $i$ is one of the whole numbers in that range.

Example

i \in \{0, 1, \dots, |V|-1\}

i is a token id: a whole number from 0 up to one less than the vocabulary size.

Where the course uses it

1.1 Scalars, Vectors, Matrices, and Tensors · Module 1, Shapes and Linear Algebrafirst use
1.5 Embeddings: From a Token to a Vector · Module 1, Shapes and Linear Algebra
1.6 Linear Projections: One Vector Into Three · Module 1, Shapes and Linear Algebra
6.1 The Question Attention Answers · Module 6, Attention
6.6 Multi-Head Attention · Module 6, Attention
7.4 The Feed-Forward Network · Module 7, The Block and the Full Model
7.5 The Output Layer · Module 7, The Block and the Full Model
8.1 The 2017 Encoder-Decoder and Cross-Attention · Module 8, Three Families: Encoder, Encoder-Decoder, Decoder
8.2 Encoder-Only: BERT and Masked Language Modeling · Module 8, Three Families: Encoder, Encoder-Decoder, Decoder
12.2 The KV Cache · Module 12, Generation
12.3 Decoding: From Logits to Text · Module 12, Generation
15.1 Rotary Position Embedding (RoPE) · Module 15, The LLaMA Block
17.2 Many Small Experts · Module 17, Mixture of Experts
24.2 Uniform Integer Quantization · Module 24, Quantization and Number Formats
25.2 The Vision Transformer · Module 25, Vision Transformers and Contrastive Learning
26.1 Connecting Vision to Language · Module 26, Vision-Language Models
26.4 Training on Images and Text Together · Module 26, Vision-Language Models
27.3 The CTC Objective and Its Recursion · Module 27, Recognition: OCR and Speech With CTC
28.3 Vector Quantization and Residual VQ · Module 28, Generating Audio: Codecs and TTS

Related terms

ℝ^d — all vectors of length dRead "R d" or "R to the d". It is the set of all lists of d real numbers, so "x ∈ ℝ^d" says x is a vector of d numbers.
{ } — a setCurly braces list the members of a set: a collection of values with no order and no repeats. Read {0, 1} as "the set of 0 and 1".
|V| — size of the vocabularyRead "the size of V". Single bars around a set mean "how many items it has", so |V| is the number of tokens in the vocabulary.