Llama 3.1

All models, with their blocks →

Size

405B

Published
Meta · Jul 2024
Line
Dense decoder · follows Llama 3
View
Source

The Llama 3 Herd of Models (arXiv 2407.21783v3) · DPO (arXiv 2305.18290v3) · no training code is publishedHow to read

DPO step

DPO step

Mask formatting tokens

×

no parameters

The special formatting tokens, including header and termination tokens, are left out of the loss in both responses. The paper hypothesizes that, being in both, they would be pushed up and down at once.

in
yy [B batch, 2, T positions]
out
mm [B batch, 2, T positions]

Meta’s change, from the Llama 3 paper (p. 16).

Sources1

Direct evidence

§4.1.4, Direct Preference Optimization: DPO on the SFT models with learning rate 10⁻⁵ and β = 0.1; special formatting tokens, including header and termination tokens, are masked out of the loss in chosen and rejected responses; an NLL term with scaling coefficient 0.2 is added on the chosen sequences.

meta · arXiv 2407.21783 / Llama3-Herd-2407.21783v3.pdf
Pinned revision v3 ↗ · p. 16