Llama 3.1

All models, with their blocks →

Size

405B

Published
Meta · Jul 2024
Line
Dense decoder · follows Llama 3
View
Source

The Llama 3 Herd of Models (arXiv 2407.21783v3) · DPO (arXiv 2305.18290v3) · no training code is publishedHow to read

DPO step

DPO step

Add, 0.2 on the NLL

×

no parameters

The two terms together, the NLL with coefficient 0.2.

L=LDPO+0.2 LNLL\mathcal L=\mathcal L_{\mathrm{DPO}}+0.2\,\mathcal L_{\mathrm{NLL}}
in
LDPO\mathcal L_{\mathrm{DPO}} [1]LNLL\mathcal L_{\mathrm{NLL}} [1]
out
L\mathcal L [1]
Sources1

Direct evidence

§4.1.4, Direct Preference Optimization: DPO on the SFT models with learning rate 10⁻⁵ and β = 0.1; special formatting tokens, including header and termination tokens, are masked out of the loss in chosen and rejected responses; an NLL term with scaling coefficient 0.2 is added on the chosen sequences.

meta · arXiv 2407.21783 / Llama3-Herd-2407.21783v3.pdf
Pinned revision v3 ↗ · p. 16