Kimi K3

Size

2.8T

Published

Moonshot AI · Release date not pinned

Line

New parts in the block · after Kimi K2

View

Source

Kimi K3 technical report @ 3cb39df · Kimi K2.5 report (arXiv 2602.02276v2), for the RL step · no training code is publishedHow to read

MOPD reward

x Queryx x e[B]Student writes

y\sim\pi_\theta(\cdot\mid e,x)

y[B, T]Teacher (d, e)

p_T=\pi^{(d,e)}_{\mathrm{teacher}}(y_t\mid x,y_{<t})

p_T[B, T]Student

p_S=\pi_\theta(y_t\mid e,x,y_{<t})

p_S[B, T]Clipped log-ratio

r_t=\operatorname{clip}\big(\operatorname{sg}(\log(p_T/p_S)),\pm {\htmlClass{vl-stream-context}{R_{\max}}}\big)

r[B, T]hatched band: not updated in this step · solid band: what this stage trains · dashed frame: concluded, not stated by the reportx: one query, named beside each model that reads it · the update that uses r is outside this plate

MOPD reward

Per-token reward

tensor

One reward for each token of the response: a dense signal for the RL framework. The update that uses it is not written in this report.

shape: $r$ [B batch, T positions]

Sources1

Contextual evidence

§4.1.2, Algorithm: policy optimization “follows the algorithm in Kimi K2.5”, with a per-token regularization; this report does not write it.

rl · MoonshotAI/Kimi-K3 / k3_tech_report.pdf
Pinned revision 3cb39dfd32e5 ↗ · p. 13