Scalar — one number
A single number, with no axes. The loss, the learning rate and the temperature are scalars. Written as a plain italic letter, such as x.
A scalar has no axes: there is nothing to index, because there is only one value. Writing says is one real number.
In PyTorch a scalar is a tensor of shape (), an empty shape. Multiplying a vector or matrix by a scalar scales every entry by the same amount.
Example
A temperature, a learning rate and a loss: each is one number.
Where the course uses it
Related terms
- Vector — a list of numbersAn ordered list of numbers, with one axis. One index picks out one entry. Each token becomes a vector before the model can compute with it.
- ℝ — the real numbersRead "R" or "the reals". It is the set of all ordinary numbers: whole, negative, fractions and decimals.
- Bold letters are vectorsA bold lowercase letter, like x in bold, is a vector (a list of numbers). A plain italic letter is one number. A capital letter is a matrix.