Matrix multiplication
Two matrices written side by side, like AB, are multiplied. Each entry of the result is a row of A dotted with a column of B. An (m×n) times an (n×p) gives an (m×p).
Entry of is the dot product of row of with column of . For that to work, a row of must be as long as a column of , so the inner sizes must match. Two things to remember:
- Shapes tell you what happens. , with of width and of shape , produces a query of width . The shared middle size disappears.
- Order matters. and are usually different, and one of them may not even be defined.
Every "projection" in the course is one matrix multiplication. In PyTorch it is A @ B.
Example
A 1×2 times a 2×2 gives a 1×2.
Where the course uses it
- 1.1 Scalars, Vectors, Matrices, and Tensors · Module 1, Shapes and Linear Algebrafirst use
- 1.6 Linear Projections: One Vector Into Three · Module 1, Shapes and Linear Algebra
- 6.2 Scaled Dot-Product Attention, by Hand · Module 6, Attention
- 6.6 Multi-Head Attention · Module 6, Attention
- 7.4 The Feed-Forward Network · Module 7, The Block and the Full Model
- 7.6 The Complete Model · Module 7, The Block and the Full Model
- 11.4 Tensor Parallelism · Module 11, Training Across Many GPUs
- 12.2 The KV Cache · Module 12, Generation
- 19.3 Estimating Parameters, Cache, and Compute · Module 19, Reading a Model Card
Related terms
- TransposeRead "A transpose". The raised T flips a matrix so its rows become its columns. On a vector it turns a column into a row.
- A matrix written outSquare brackets around a grid of numbers show a matrix entry by entry. Each line is a row; each position across a line is a column.
- Query, key, valueThree learned versions of each token's vector. The query is what the token is looking for, the key is what it offers to others, and the value is what it passes on. W^Q is read "W Q", the matrix that makes queries.