- out
- [B batch, T positions]
Explanation
The teacher is one of the nine experts: the one of the query’s domain and of the effort sampled for it. In eq. 15 the student’s probability is conditioned on the effort and the teacher’s is not; the report does not say this in words.
The plate stops at the reward. The report says this dense reward “seamlessly integrates into our RL framework”, partial rollouts included; it does not write the update that uses it, nor say whether it is the only reward there. For the RL stage it says policy optimization follows the algorithm in Kimi K2.5.
, the checkpoint the student starts from, the number of steps and the learning rate are not disclosed. A distillation objective over the top-k tokens was tried and gave no clear advantage.
Sources4
Direct evidence
§4.1.3, Multi-Teacher On-Policy Distillation: for a domain d and a sampled reasoning effort e ∈ {low, high, max}, optimization is guided by the teacher of that pair among the nine experts. The per-token reward on y_t is the stop-gradient log-ratio of the teacher’s probability to the student’s, clipped to ±R_max (eq. 15). The dense reward integrates into the RL framework, partial rollouts included.
mopd · MoonshotAI/Kimi-K3 / k3_tech_report.pdf
Pinned revision 3cb39dfd32e5 ↗ · pp. 13–14
Direct evidence
§4.1.2: three domains crossed with three reasoning efforts in {low, high, max} yield nine expert models.
experts · MoonshotAI/Kimi-K3 / k3_tech_report.pdf
Pinned revision 3cb39dfd32e5 ↗ · p. 12
Direct evidence
§4.1.2, Algorithm: policy optimization “follows the algorithm in Kimi K2.5”, with a per-token regularization; this report does not write it.
rl · MoonshotAI/Kimi-K3 / k3_tech_report.pdf
Pinned revision 3cb39dfd32e5 ↗ · p. 13
Direct evidence
§4.1.2, Reasoning Effort RL: “Trajectories produced by the resulting experts at all reasoning levels are jointly collected for supervised fine-tuning and multi-teacher on-policy distillation.”
trajectories · MoonshotAI/Kimi-K3 / k3_tech_report.pdf
Pinned revision 3cb39dfd32e5 ↗ · p. 13