R^\top R = I

Orthogonal matrix

A matrix that only rotates (or mirrors) vectors: it keeps every length and every angle, and its transpose undoes it. R^⊤R = I says exactly that.

Because $R^\top = R^{-1}$ (the transpose is the inverse), the RoPE identity $R_m^\top R_n = R_{n-m}$ follows in one line: undo a rotation by $m$ , then apply a rotation by $n$ , and the net rotation is by $n - m$ . And because rotations keep lengths, RoPE cannot make a query or key longer or shorter. It only turns it, so position is encoded without changing the vector's size.

Example

R = \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}: \; R[3, 4]^\top = [-4, 3]^\top

A 90° rotation. Both vectors have length 5.

Where the course uses it

15.1 Rotary Position Embedding (RoPE) · Module 15, The LLaMA Blockfirst use

Related terms

RoPE rotation matrixRead "R theta m". It rotates a query or key vector by an angle that grows with the token's position m, so position is stored as an amount of rotation.
TransposeRead "A transpose". The raised T flips a matrix so its rows become its columns. On a vector it turns a column into a row.
I — the identity matrixRead "I" or "the identity". It is a square matrix with 1s on the diagonal and 0s elsewhere. Multiplying by it leaves a vector unchanged.