Training

  1. Random initialisation
  2. Pre-trainingDeepSeek-V314.8T
  3. Context 32KDeepSeek-V3≈62.9B
  4. Context 128KDeepSeek-V3≈62.9B
  5. DeepSeek-V3-Base
  6. SFT → RLDeepSeek-V3tokens not disclosed
  7. DeepSeek-V3
  8. From V3’s base: before or after its context extension, not stated
  9. Context 32KDeepSeek-V3.1630B
  10. Context 128KDeepSeek-V3.1209B
  11. DeepSeek-V3.1-Base
  12. Post-trainingDeepSeek-V3.1tokens not disclosed
  13. DeepSeek-V3.1
  14. From V3.1 or from a base checkpoint: not stated
  15. UpdateDeepSeek-V3.1-Terminustokens not disclosed
  16. DeepSeek-V3.1-Terminus
  17. V3.1-Base or a later checkpoint: not stated
  18. V3.1-Terminus base checkpoint
  19. Dense warm-upDeepSeek-V3.2-Exp2.1B
  20. Sparse trainingDeepSeek-V3.2-Exp943.7B
  21. DeepSeek-V3.2 base
  22. Specialist distillationDeepSeek-V3.2-Exptokens not disclosed
  23. Mixed RLDeepSeek-V3.2-Exptokens not disclosed
  24. DeepSeek-V3.2-Exp

Random initialisation

×

Checkpoint

Every learnable parameter drawn with standard deviation 0.006.

Weights
no pinned source shows them published
Sources1
Key
stage
tokens, log scale
a total no source states, computed from steps × batch × length
checkpoint
checkpoint, weights published
a join the sources leave unexplained
stages of a released model, folded into one row
derived
computed from reported values; the arithmetic is shown
not disclosed
no pinned source states it; nothing is filled in

DeepSeek-V3.2-Exp

All models, with their blocks →

Size
671B
Published
DeepSeek · Sep 2025
Line
Latent attention and experts
View