Hyperparameter — a setting chosen by hand
A setting fixed before training rather than learned: the learning rate, batch size, number of layers, weight decay, dropout rate and so on.
Chosen by comparing validation loss across candidate values (section 4.5).