SFT
×Stage · DeepSeek-V3
Two epochs on 1.5M instruction instances.
- Data
- 1.5M instances across domains. Reasoning data first comes from an internal DeepSeek-R1 model; expert models trained with SFT and RL blend its responses with concise ones, and rejection sampling curates the result. Non-reasoning data comes from DeepSeek-V2.5, checked by annotators
- Epochs
- 2
- Learning rate
- Cosine, 5×10⁻⁶ → 1×10⁻⁶
- Packing
- Several samples per sequence, kept apart by sample masking
Not disclosed Batch · Optimiser
Sources2
- DeepSeek-V3 report, pp. 28–29Data
- DeepSeek-V3 report, p. 29Epochs, Learning rate, Packing
Key
- stage
- tokens, log scale
- a total no source states, computed from steps × batch × length
- checkpoint
- checkpoint, weights published
- a join the sources leave unexplained
- stages of a released model, folded into one row
- derived
- computed from reported values; the arithmetic is shown
- not disclosed
- no pinned source states it; nothing is filled in