\mathbf{x} \in \mathbb{R}^{n}

Vector — a list of numbers

An ordered list of numbers, with one axis. One index picks out one entry. Each token becomes a vector before the model can compute with it.

A vector has one axis, and its length is how many numbers it holds. $\mathbf{x} \in \mathbb{R}^{4}$ is a vector of 4 numbers, and $x_2$ is its second entry.

The course writes vectors in bold and, following most transformer code, as rows: a token vector sits on the left of a matrix, as in $\mathbf{x} W$ . Geometrically a vector is an arrow from the origin, which is what gives it a length and a direction (section 1.5).

In PyTorch a vector of 4 numbers has shape (4,).

Example

\mathbf{x} = [0.2,\, -1.7,\, 3.0,\, 0.5] \;\Longrightarrow\; x_3 = 3.0

One axis of length 4. The third entry is 3.0.

Where the course uses it

1.1 Scalars, Vectors, Matrices, and Tensors · Module 1, Shapes and Linear Algebrafirst use

Related terms

Scalar — one numberA single number, with no axes. The loss, the learning rate and the temperature are scalars. Written as a plain italic letter, such as x.
Matrix — a table of numbersA rectangular table of numbers, with two axes: rows and columns. An m×n matrix has m rows and n columns. Written as a capital letter, such as W.
Bold letters are vectorsA bold lowercase letter, like x in bold, is a vector (a list of numbers). A plain italic letter is one number. A capital letter is a matrix.
ℝ^d — all vectors of length dRead "R d" or "R to the d". It is the set of all lists of d real numbers, so "x ∈ ℝ^d" says x is a vector of d numbers.