Llama 3.1

All models, with their blocks →

Size

405B

Published
Meta · Jul 2024
Line
Dense decoder · follows Llama 3
View
Source

The Llama 3 Herd of Models (arXiv 2407.21783v3) · DPO (arXiv 2305.18290v3) · no training code is publishedHow to read

DPO step

DPO step

Chosen and rejected

×

tensor

The chosen response ywy_w and the rejected one yly_l of a preference pair: only pairs rated significantly better or better.

shape
yy [B batch, 2, T positions]
Sources1

Contextual evidence

§4.1.4 and §4.2.1: mostly the latest batches of preference data, collected with the best models of the previous rounds; only samples whose chosen response is rated significantly better or better than the rejected one.

prefs · arXiv 2407.21783 / Llama3-Herd-2407.21783v3.pdf
Pinned revision v3 ↗ · pp. 16–17