DeepSeek V4.1

Size

552B + 196B Engram

Published

DeepSeek · Sep 2026

Line

New parts in the block · after DeepSeek-V3.2-Exp

View

Source

DeepSeek‑V4.1 technical report @ 2cba9e4 · no training code is publishedHow to read

Effort-conditioned reward

x[B]b[|𝔅|]Policy

z_{b,j}\sim\pi_\theta(\cdot\mid x,b)

z[B, |𝔅|, M]Reward r[B, |𝔅|, M]Penalty coefficient

{\htmlClass{vl-stream-k}{k}}(b)={\htmlClass{vl-stream-k}{k_0}}\,e^{-(b-b_{\min})/\tau}

k[|𝔅|]Add length penalty

{\htmlClass{vl-stream-context}{R}}=r-\min\{C_{\max},{\htmlClass{vl-stream-k}{k}}(b)\,\ell/L_{\mathrm{norm}}\}

R[B, |𝔅|, M]Centre in the subgroup A[B, |𝔅|, M]solid band: what this stage trainsthe update that uses A is outside this plate

Effort-conditioned reward

Reward before the penalty

tensor

$r$ , this plate’s name: one number for each response, before the length penalty.

shape: $r$ [B batch, |𝔅|, M]

Sources

Contextual evidence from Reward

§5.1.4, Controllable Reasoning Effort in RL: a scalar effort b ∈ {1, …, 100} is prepended to the system prompt. For each prompt x, M_b responses are sampled at each effort level b ∈ B (eq. 8). Responses sharing (x, b) form a subgroup, within which rewards are mean-centered to compute group-relative advantages. A length-penalty term is added to the reward: −min{C_max, k(b)·ℓ/L_norm} (eq. 9), with k(b) = k₀·exp(−(b − b_min)/τ), τ = λ·Δb̄, Δb̄ the average spacing between the training effort levels (eq. 10).

effort · deepseek-ai/DeepSeek-V4.1-Flash / DeepSeek_V41_Tech_Report.pdf
Pinned revision 2cba9e42aa02 ↗ · p. 29

Contextual evidence from Reward

§5.1.1: “We formalize each task as a triplet (problem, environment, verification system)”.

tasks · deepseek-ai/DeepSeek-V4.1-Flash / DeepSeek_V41_Tech_Report.pdf
Pinned revision 2cba9e42aa02 ↗ · p. 25