LDLMIn construction

All models, with previews →

Meshchaninov et al. · May 2026Diffusion language model

Encoder block

Pinned implementation and paper LDLM @ e18c42a, not yet public · arXiv:2605.07933v1 · OpenWebText · 1024 · checkpoint, tokenizer and saved statistics not verified · How to read

TrainingGeneration

Latent encoder › 6 encoder blocks

Add MLP update

×

no parameters

Add the mechanism’s update to the residual stream element by element.

un+1=un′+Δmlp{\htmlData{light=block-output mlp-add}{u_{n+1}}}={\htmlData{light=residual attention-add}{u'_n}}+{\htmlData{light=mlp}{\Delta_{\mathrm{mlp}}}}
in
un′u'_n [B batch, T positions, 768 d]Δmlp\Delta_{\mathrm{mlp}} [B batch, T positions, 768 d]
out
un+1u_{n+1} [B batch, T positions, 768 d]
Sources1

Direct evidence

An encoder block adds the Perceiver attention’s output to the latents, then adds the MLP’s output.

The implementation, pinned at revision e18c42acdfb7, not yet public