DeepSeek-V3.2-Exp

All models, with previews →

DeepSeek · Sep 2025Latent attention and experts

Lightning indexer

Reference code deepseek-ai/DeepSeek-V3.2-Exp @ 87e509a · checkpoint, tokenizer and tensors not verified · How to read

Full sequencePrefillDecode

61 transformer blocks › 58 MoE blocks › Sparse latent attention › Lightning indexer

Index key cache

×

state · kept between callsself.k_cache, self.k_scale_cache

The indexer key of every position so far, in FP8 with one float32 scale per 128 features, kept between forward calls.

shape
kˉI\bar k^{I} k_I [B batch, 1001 cached positions, 128 index d_h]

This call writes row 1,000 and scores the new query against rows 0–1,000. Each position holds 128 FP8 features and their scale, allocated for 16,384 positions per sequence slot (model.py 56).

Codemodel.py 56–480
model.py 453–454
        self.register_buffer("k_cache", torch.zeros(args.max_batch_size, args.max_seq_len, self.head_dim, dtype=torch.float8_e4m3fn), persistent=False)
        self.register_buffer("k_scale_cache", torch.zeros(args.max_batch_size, args.max_seq_len, self.head_dim // block_size, dtype=torch.float32), persistent=False)
model.py 476–480
        self.k_cache[:bsz, start_pos:end_pos] = k_fp8
        self.k_scale_cache[:bsz, start_pos:end_pos] = k_scale
        weights = self.weights_proj(x.float()) * self.n_heads ** -0.5
        weights = weights.unsqueeze(-1) * q_scale * self.softmax_scale
        index_score = fp8_index(q_fp8.contiguous(), weights, self.k_cache[:bsz, :end_pos].contiguous(), self.k_scale_cache[:bsz, :end_pos].contiguous())
model.py 56
    max_seq_len: int = 4096 * 4
model.py 81–86
    original_seq_len: int = 4096
    rope_theta: float = 10000.0
    rope_factor: float = 40
    beta_fast: int = 32
    beta_slow: int = 1
    mscale: float = 1.
Sources2

Contextual evidence

The indexer keeps its own FP8 key cache with one float32 scale per 128 features; each call writes its keys and scores its queries against every cached position so far.

indexer-cache · deepseek-ai/DeepSeek-V3.2-Exp / inference/model.py
Pinned revision 87e509a2e5a1 ↗ · lines 453–480

Read code excerpt
        self.register_buffer("k_cache", torch.zeros(args.max_batch_size, args.max_seq_len, self.head_dim, dtype=torch.float8_e4m3fn), persistent=False)
        self.register_buffer("k_scale_cache", torch.zeros(args.max_batch_size, args.max_seq_len, self.head_dim // block_size, dtype=torch.float32), persistent=False)

… lines 455–475 omitted …

        self.k_cache[:bsz, start_pos:end_pos] = k_fp8
        self.k_scale_cache[:bsz, start_pos:end_pos] = k_scale
        weights = self.weights_proj(x.float()) * self.n_heads ** -0.5
        weights = weights.unsqueeze(-1) * q_scale * self.softmax_scale
        index_score = fp8_index(q_fp8.contiguous(), weights, self.k_cache[:bsz, :end_pos].contiguous(), self.k_scale_cache[:bsz, :end_pos].contiguous())

Contextual evidence

Reference defaults the configuration does not override: maximum sequence length, YaRN RoPE settings and the softmax scale correction.

model-args · deepseek-ai/DeepSeek-V3.2-Exp / inference/model.py
Pinned revision 87e509a2e5a1 ↗ · lines 56–86

Read code excerpt
    max_seq_len: int = 4096 * 4

… lines 57–80 omitted …

    original_seq_len: int = 4096
    rope_theta: float = 10000.0
    rope_factor: float = 40
    beta_fast: int = 32
    beta_slow: int = 1
    mscale: float = 1.