LDLMIn construction

All models, with their blocks →

Size
306M trained
Published
Meshchaninov et al. · May 2026
Line
Diffusion language model
View
Architecture
Source

LDLM @ e18c42a, not yet public · arXiv:2605.07933v1 · OpenWebText · 1024 · checkpoint, tokenizer and saved statistics not verifiedHow to read

Denoiser

TrainingGeneration

Sampler › Denoiser

Input projection

×

learned

Map the 1,536 joined features to the denoiser’s 768.

x0=[ zt  z^0sc ]Win+bin{\htmlData{light=embedded input-projection}{x_0}}=[\,z_t\;\hat z_0^{\mathrm{sc}}\,]W_{\mathrm{in}}+b_{\mathrm{in}}
in
[ zt  z^0sc ][\,z_t\;\hat z_0^{\mathrm{sc}}\,] [B batch, T positions, 1536]
out
x0x_0 [B batch, T positions, 768 d]
parameters
1,180,416
Parameters

1,180,416 parameters per instance.

Exact per-instance accounting
ComponentCalculationParameters
Weight matrix1,536×7681{,}536 \times 7681,179,648
Output bias768768768
Per-instance total1,180,416

The reference module holds the weight output by input, [out, in].

Across 1024 shared calls: 1,180,416×1=1,180,4161{,}180{,}416 \times 1 = 1{,}180{,}416 parameters.

Derived from the configured matrix dimensions; checkpoint tensors have not been independently counted.

Sources2

Direct evidence

The denoiser owns the time embedding’s two affine layers, one rotary table for all blocks, an affine projection of the concatenated input, and the stack.

The implementation, pinned at revision e18c42acdfb7, not yet public

Direct evidence

The time embedding passes its MLP and a SiLU; the noised latents are concatenated with the previous clean estimate, zeros when there is none, and projected; the stack runs in bfloat16.

The implementation, pinned at revision e18c42acdfb7, not yet public