DeepSeek-V3.2-Exp

DeepSeek · Sep 2025Latent attention and experts

Reference code deepseek-ai/DeepSeek-V3.2-Exp @ 87e509a · checkpoint, tokenizer and tensors not verified · How to read

Raw text Tokenizer datafixed assets TokenizerChat template Token IDs[B, T]Token embedding

E

129,280 × 7,168 x₀[B, T, 7168]Position IDs

0,\ldots,T-1

Generate RoPE coefficients

{\htmlClass{vl-stream-context}{R_{t,j}}}=e^{\mathrm i t\omega_j}

R[T, 32]Construct causal mask

{\htmlClass{vl-stream-context}{M_{t,s}}}=\begin{cases}0&s\le t\\-\infty&s>t\end{cases}

M[T, T]

3 dense Open →

58 MoE Open →

x₆₁ Final RMSNorm

\operatorname{mean}(x^2)

learned gain Vocabulary projection

L=zW_{\mathrm{out}}

7,168 × 129,280 Logits[B, T, 129280]

Raw text

tensorprompts

Text supplied for tokenization.

Codegenerate.py 117, 150

generate.py 117

    tokenizer = AutoTokenizer.from_pretrained(ckpt_path)

generate.py 150

        prompt_tokens = [tokenizer.apply_chat_template([{"role": "user", "content": prompt}], add_generation_prompt=True) for prompt in prompts]

Sources1

Contextual evidence

The reference generator loads the checkpoint’s tokenizer with Hugging Face AutoTokenizer and encodes each prompt through its chat template. The tokenizer’s vocabulary and rules ship with the checkpoint; these sources do not include them.

tokenizer · deepseek-ai/DeepSeek-V3.2-Exp / inference/generate.py
Pinned revision 87e509a2e5a1 ↗ · lines 117–150

Read code excerpt

    tokenizer = AutoTokenizer.from_pretrained(ckpt_path)

… lines 118–149 omitted …

        prompt_tokens = [tokenizer.apply_chat_template([{"role": "user", "content": prompt}], add_generation_prompt=True) for prompt in prompts]