Training

  1. Random initialisation
  2. Pre-trainingDeepSeek-V314.8T
  3. Context 32KDeepSeek-V3≈62.9B
  4. Context 128KDeepSeek-V3≈62.9B
  5. DeepSeek-V3-Base
  6. SFT → RLDeepSeek-V3tokens not disclosed
  7. DeepSeek-V3
  8. From V3’s base: before or after its context extension, not stated
  9. Context 32KDeepSeek-V3.1630B
  10. Context 128KDeepSeek-V3.1209B
  11. DeepSeek-V3.1-Base
  12. Post-trainingDeepSeek-V3.1tokens not disclosed
  13. DeepSeek-V3.1
  14. From V3.1 or from a base checkpoint: not stated
  15. UpdateDeepSeek-V3.1-Terminustokens not disclosed
  16. DeepSeek-V3.1-Terminus
  17. V3.1-Base or a later checkpoint: not stated
  18. V3.1-Terminus base checkpoint
  19. Dense warm-upDeepSeek-V3.2-Exp2.1B
  20. Sparse trainingDeepSeek-V3.2-Exp943.7B
  21. DeepSeek-V3.2 base
  22. Specialist distillationDeepSeek-V3.2-Exptokens not disclosed
  23. Mixed RLDeepSeek-V3.2-Exptokens not disclosed
  24. DeepSeek-V3.2-Exp

From V3’s base: before or after its context extension, not stated

×

Join the sources leave unexplained

The V3.1 card starts from “the original V3 base checkpoint” and does not say whether before or after V3’s own context extension.

Sources1
Key
stage
tokens, log scale
a total no source states, computed from steps × batch × length
checkpoint
checkpoint, weights published
a join the sources leave unexplained
stages of a released model, folded into one row
derived
computed from reported values; the arithmetic is shown
not disclosed
no pinned source states it; nothing is filled in

DeepSeek-V3.2-Exp

All models, with their blocks →

Size
671B
Published
DeepSeek · Sep 2025
Line
Latent attention and experts
View