78 transformer blocks › 3 dense blocks › Sparse latent attention › Lightning indexer
Index key cache
×state · kept between callspast_key_values.update_indexer(k, self.layer_idx)
The indexer key of every position so far, kept between forward calls.
- shape
- k_I [B batch, T cached positions, 128 index d_h]
Prefill hands the prompt’s keys to the cache and scores against them. 128 features per position.
Codemodeling_glm_moe_dsa.py 236–239
if past_key_values is not None:
k = past_key_values.update_indexer(k, self.layer_idx)
scores = torch.matmul(q.float(), k.transpose(-1, -2).float().unsqueeze(1)) * self.softmax_scaleSources1
Contextual evidence
The indexer hands its key to the cache and scores its queries against every cached key so far.
indexer-cache · huggingface/transformers / modeling_glm_moe_dsa.py
Pinned revision 5e4d6304de55 ↗ · lines 236–239
Read code excerpt
if past_key_values is not None:
k = past_key_values.update_indexer(k, self.layer_idx)
scores = torch.matmul(q.float(), k.transpose(-1, -2).float().unsqueeze(1)) * self.softmax_scale