DeepSeek-V4.1-Flash · training steps · Penalty coefficient

Effort-conditioned reward

Penalty coefficient

no parameters

How heavily a token of reasoning is charged at this effort. It falls exponentially as the effort rises: raising $b$ by $\tau$ divides it by $e$ .

{\htmlData{light=k coefficient}{{\htmlClass{vl-stream-k}{k}}}}({\htmlData{light=effort}{b}})={\htmlClass{vl-stream-k}{k_0}}\exp\!\Big(-\frac{{\htmlData{light=effort}{b}}-b_{\min}}{\tau}\Big),\qquad\tau=\lambda\,\overline{\Delta {\htmlData{light=effort}{b}}}

in: $b$ [|𝔅|]
out: $k$ [|𝔅|]

Explanation

$k_0$ is the coefficient at the lowest trained effort $b_{\min}$ , and $\overline{\Delta b}$ the average spacing between the trained levels. A smaller $\tau$ tends to separate the efforts’ behaviour more.

Sources2

Direct evidence

§5.1.4, Controllable Reasoning Effort in RL: a scalar effort b ∈ {1, …, 100} is prepended to the system prompt. For each prompt x, M_b responses are sampled at each effort level b ∈ B (eq. 8). Responses sharing (x, b) form a subgroup, within which rewards are mean-centered to compute group-relative advantages. A length-penalty term is added to the reward: −min{C_max, k(b)·ℓ/L_norm} (eq. 9), with k(b) = k₀·exp(−(b − b_min)/τ), τ = λ·Δb̄, Δb̄ the average spacing between the training effort levels (eq. 10).

effort · deepseek-ai/DeepSeek-V4.1-Flash / DeepSeek_V41_Tech_Report.pdf
Pinned revision 2cba9e42aa02 ↗ · p. 29

Direct evidence

Appendix C: k₀ is the penalty coefficient at the lowest effort level b_min.

k0 · deepseek-ai/DeepSeek-V4.1-Flash / DeepSeek_V41_Tech_Report.pdf
Pinned revision 2cba9e42aa02 ↗ · p. 49