Training

  1. Start of training
  2. Start of the encoder’s training
  3. Vision encoder: contrastiveDeepSeek-ViT≈ 47B pairsnot statedContrastiveSigLIPViT
  4. Vision encoder: autoregressiveDeepSeek-ViT236Bnot statedNTPViT
  5. DeepSeek-ViT
  6. Pre-training 0–28TDeepSeek-V4.1-Flash28T64KNTPBalanceweight 0.0001not ViTsparse from scratch
  7. Pre-training 28–34TLR decay begins6T64KNTPBalanceweight 0.0001all
  8. Pre-training 34–45TDeepSeek-V4.1-Flash11T1MNTPBalanceweight 0.0001all
  9. DeepSeek-V4.1-Flash-Base
  10. DSpark stageDeepSeek-V4.1-Flashnot statednot statednot statedDSparkDSpark added
  11. SFTDeepSeek-V4.1-Flashnot statednot statedSFTnot statedpost-training changes
  12. RLDeepSeek-V4.1-Flashnot statednot statedRLgroup-relativenot stated
  13. On-policy distillationDeepSeek-V4.1-Flashnot statednot statedOPDover 40 teachersnot stated
  14. DeepSeek-V4.1-Flash
  • tokens, log scale
  • derived: computed from stated numbers
  • context length
  • all parameters train
  • a part trains
  • all but a part trains
  • NTPdashed: concluded, no source names it
  • not stated by the sources
  • the architecture changes
  • checkpoint
  • checkpoint, weights published
  • a join the sources leave unexplained

DeepSeek-ViT

×

Checkpoint

The vision encoder after its two stages: the 4B LLM it was fine-tuned with is discarded and the encoder kept for pre-training.

Weights
no pinned source shows them published
Sources1
Key
stage
tokens, log scale
a total no source states, computed from steps × batch × length
checkpoint
checkpoint, weights published
a join the sources leave unexplained
stages of a released model, folded into one row
derived
computed from reported values; the arithmetic is shown
not disclosed
no pinned source states it; nothing is filled in

DeepSeek V4.1

All models, with their blocks →

Size
552B + 196B Engram
Published
DeepSeek · Sep 2026
Line
New parts in the block · after DeepSeek-V3.2-Exp
View
Source

DeepSeek‑V4.1 technical report @ 2cba9e4 · no training code is publishedHow to read