Tensor — numbers on any number of axes
An array of numbers with any number of axes. In code every array is called a tensor: a scalar has 0 axes, a vector 1, a matrix 2. In prose the word usually means 3 or more.
Each extra axis is one more index. A stack of matrices is a 3-axis tensor: a batch of sentences, each tokens by numbers, has shape . A color image is another: height by width by 3 color channels (section 25.2).
Multiply the sizes to count the numbers: a tensor holds 24.
Two meanings of rank. Some texts call the number of axes a tensor's rank or order. Section 20.2 uses rank for something else, the number of independent directions in a matrix. This course says "number of axes" to keep them apart; PyTorch calls it ndim.
Example
Two sentences, three tokens each, four numbers per token. PyTorch shape (2, 3, 4).
Where the course uses it
Related terms
- Matrix — a table of numbersA rectangular table of numbers, with two axes: rows and columns. An m×n matrix has m rows and n columns. Written as a capital letter, such as W.
- Shape and axesA tensor's shape lists the size of each axis, in order. (2, 3, 4) has three axes of sizes 2, 3 and 4. Checking shapes is the quickest way to follow a formula.
- Batch — examples processed togetherA group of examples processed at the same time. It adds one axis, of size B, at the front of every tensor. Every example in the batch goes through the same computation.
- ℝ^d — all vectors of length dRead "R d" or "R to the d". It is the set of all lists of d real numbers, so "x ∈ ℝ^d" says x is a vector of d numbers.