Perplexity — exp of the average loss
The exponential of the average negative log-likelihood per token. A perplexity of 20 means the model is as unsure as if choosing uniformly among 20 tokens.
Lower is better. It is per token, so it cannot be compared across tokenizers; bits per byte can (section 13.1).