LDLMIn construction

Meshchaninov et al. · May 2026Diffusion language model

Latent decoder

Pinned implementation and paper LDLM @ e18c42a, not yet public · arXiv:2605.07933v1 · OpenWebText · 1024 · checkpoint, tokenizer and saved statistics not verified · How to read

Training Generation

z₀[B, T, 768]Position tableE_p E_p Positions to latents

z^{p}=\tilde z_0+E_{p}

z^p[B, T, 768]Mask vector

E[\texttt{<mask>}]

Query MLP Open →

Scale

(1+s_e)

Positions to queries

r_0=\tilde e+E_{p}

r₀[B, T, 768]

6 decoder blocks Open →

h[B, T, 768]

Latent decoder

Positions to queries

no parameters

Add each position’s learned vector to its query, from the same table as the latents’.

{\htmlData{light=start add-query-positions}{r_0}}={\htmlData{light=query-scale}{\tilde e}}+{\htmlData{light=positions}{E_{p}}}

in: $\tilde e$ [B batch, T positions, 768 d] $E_{p}$ [B batch, T positions, 768 d]
out: $r_0$ [B batch, T positions, 768 d]

Sources1

Direct evidence

Every query is the mask vector: it passes the MLP and a scale, positions are added; the latents take the same position table; the blocks are applied; every position counts as masked.

The implementation, pinned at revision e18c42acdfb7, not yet public