Training

  1. Start of training
  2. Start of the encoder’s training
  3. Vision encoder: contrastiveDeepSeek-ViT≈ 47B pairsnot statedContrastiveSigLIPViT
  4. Vision encoder: autoregressiveDeepSeek-ViT236Bnot statedNTPViT
  5. DeepSeek-ViT
  6. Pre-training 0–28TDeepSeek-V4.1-Flash28T64KNTPBalanceweight 0.0001not ViTsparse from scratch
  7. Pre-training 28–34TLR decay begins6T64KNTPBalanceweight 0.0001all
  8. Pre-training 34–45TDeepSeek-V4.1-Flash11T1MNTPBalanceweight 0.0001all
  9. DeepSeek-V4.1-Flash-Base
  10. DSpark stageDeepSeek-V4.1-Flashnot statednot statednot statedDSparkDSpark added
  11. SFTDeepSeek-V4.1-Flashnot statednot statedSFTnot statedpost-training changes
  12. RLDeepSeek-V4.1-Flashnot statednot statedRLgroup-relativenot stated
  13. On-policy distillationDeepSeek-V4.1-Flashnot statednot statedOPDover 40 teachersnot stated
  14. DeepSeek-V4.1-Flash
  • tokens, log scale
  • derived: computed from stated numbers
  • context length
  • all parameters train
  • a part trains
  • all but a part trains
  • NTPdashed: concluded, no source names it
  • not stated by the sources
  • the architecture changes
  • checkpoint
  • checkpoint, weights published
  • a join the sources leave unexplained

How DeepSeek-V4.1-Flash was made: the stages on its path, top to bottom, and what changes from one to the next.

Of the 9 stages (7 on the path, 2 off it), the sources give or let us compute tokens for 4, and state or let us conclude the context length for 3, the objective for 8 and what is trained for 6; what is concluded is drawn dashed. The rest is hatched: nothing is filled in.

Select a stage for its numbers, its reason and the model as that stage trains it.

DeepSeek V4.1

All models, with their blocks →

Size
552B + 196B Engram
Published
DeepSeek · Sep 2026
Line
New parts in the block · after DeepSeek-V3.2-Exp
View
Source

DeepSeek‑V4.1 technical report @ 2cba9e4 · no training code is publishedHow to read