Mixed RL
×Stage · DeepSeek-V3.2-Exp
One GRPO stage for reasoning, agents and alignment together.
- Algorithm
- GRPO in a single stage merging reasoning, agent and human-alignment training
- Rewards
- Reasoning and agents: rule-based outcome reward, length penalty and language-consistency reward; general tasks: a generative reward model with rubrics per prompt
- Attention
- Sparse, as in the sparse stage
Not disclosed Group size, steps, learning rate
Sources2
- DeepSeek-V3.2 report, p. 3Algorithm, Attention
- DeepSeek-V3.2 report, p. 4Rewards
Key
- stage
- tokens, log scale
- a total no source states, computed from steps × batch × length
- checkpoint
- checkpoint, weights published
- a join the sources leave unexplained
- stages of a released model, folded into one row
- derived
- computed from reported values; the arithmetic is shown
- not disclosed
- no pinned source states it; nothing is filled in