Contrastive learning — pull pairs together, push others apart

Contrastive learning

Contrastive learning — pull pairs together, push others apart

Training two encoders so that matching pairs (an image and its caption) score higher than every mismatched pair in the batch.

CLIP uses a softmax over the batch; SigLIP an independent sigmoid per pair (section 25.3).

Where the course uses it

25.3 Contrastive Learning: CLIP and SigLIP · Module 25, Vision Transformers and Contrastive Learningfirst use

Related terms

SoftmaxRead "softmax of z". It turns any list of numbers into probabilities: every output is positive and they add up to 1. Bigger inputs get bigger shares.
Sigmoid — a squash into (0, 1)σ(z) = 1 / (1 + e^(−z)): maps any real number into the range 0 to 1, with σ(0) = 0.5.