Skip to content

Roadmap Research Notes

This page holds the evidence behind the Roadmap. The research was done on 2026-10-03. For each item it gives:

  • Source: the primary source and its date.
  • Falsifier: the result that would tell us not to build the item, or that it doesn't work.
  • Fails when: known failure modes from papers, issue trackers and post-mortems.

Some claims about AMD's gfx1151 rest on community reports rather than AMD documentation, so they carry tags:

  • [V]: verified in a primary source (vendor docs, a spec, a paper).
  • [C]: a community report (a GitHub issue or benchmark). Real, but not reproduced by us.
  • [I]: our own inference. Treat every speedup estimate as [I] until HIP-1 measures it.

Sources and tools age quickly in this field. Check the date before relying on one.

Cross-cutting

  • Random networks give plausible explanations.
  • Méloux et al., "The Dead Salmons of AI Interpretability", arXiv 2512.18792 (2025-12): attribution, probing, sparse autoencoders and causal analyses all produce plausible output on randomly initialized networks.
  • Adebayo et al., "Sanity Checks for Saliency Maps", NeurIPS 2018: some saliency methods barely change when the model's weights are randomized.
  • Heap et al., arXiv 2501.17727 (2025-01): SAEs on randomly initialized transformers produce features that look interpretable.

This is the basis for the baseline rule and for XAI-6. - Explainers disagree with each other. Krishna et al., "The Disagreement Problem in Explainable Machine Learning", arXiv 2202.01602 (2022-02). - Surrogates can be fooled. Slack et al., AIES 2020: a model can be built to look innocent to LIME and SHAP while staying biased.

FND: Foundations

All FND items are done; see the roadmap's FND section for what changed from the plan. The notes below record the reasoning they were built on.

  • FND-1, FND-2. Module::parameters() returned std::vector<ParamRef> with no names. It is now derived from named_parameters().
  • Falsifier for freezing: a frozen parameter's bytes change after N steps, or the input gradient no longer matches finite differences.
  • FND-3, FND-4. The demand for these comes from items in other epics. FEAT-2 (Gao et al., TopK SAE, arXiv 2406.04093, 2024-06) and ARCH-3 need top-k. INT-5, FEAT-6, TRN-9 (Wu et al., ReFT, arXiv 2404.03592, 2024-04) and TRN-11 (Shuttleworth et al., arXiv 2410.21228, 2024-10) need an eigensolver or SVD. linear_algebra.hpp had only a dense solve; matrix_decompositions.hpp now has the rest.
  • Fails when: eigenvector sign and order are arbitrary, so tests must align them before comparing.
  • FND-5.
  • Sources: plans/lrp_issues.md #8. Zennit's canonizers fold BatchNorm into the preceding layer before LRP.
  • Falsifier: none. Explanations that depend on the rest of the batch are a correctness bug.
  • FND-6. VGG and ResNet are the standard LRP benchmark models, and both need stride and padding.
  • FND-7. plans/gpu_review.md lists "no atomics, so deterministic" as a current strength. Keep that.
  • FND-8. plans/gpu_review.md #1 (no device check in Module::forward) and #2 (Tensor::Stack tags the wrong device).

IO: Serialization and model import

  • IO-1.
  • Format: huggingface/safetensors (Apache-2.0) is an 8-byte length, a JSON header, then a packed data section. The header is at most 100 MB, and __metadata__ maps strings to strings.
  • Audit: Trail of Bits audited the format (published 2023-05).
  • Option: the header-only C++ reader safetensors-cpp (MIT) exists, but its README says shape validation and mmap are incomplete. Fuzz both it and a hand-written reader before choosing.
  • Fails when: an offset runs past the end of the file, ranges overlap, or the element-count product overflows.
  • IO-2. Falsifier: save, then load, doesn't give a bit-identical forward pass, or resumed training doesn't reproduce the loss curve. Both checks pass (#44).
  • IO-3: pickle is code.
  • CVE-2025-32434 (GHSA-53q9-r3pm-6pq6, 2025-04, CVSS 9.8): torch.load(weights_only=True) was remote code execution before torch 2.6.0.
  • Scanners get bypassed. "nullifAI" (ReversingLabs, 2025-02) used broken pickles and 7z-compressed .pt files; CVE-2025-46417 is another bypass; ShadowPickle (arXiv 2607.17503, 2026-07) shows evasion continues.
  • Format: torch.save writes a zip of data.pkl plus storage blobs, and tensors are rebuilt with explicit strides. Non-contiguous tensors must be made contiguous before writing.
  • IO-4: layout bugs load fine and give garbage.
  • A missing Q/K RoPE permutation gives fluent nonsense (huggingface/transformers #25199). Random-weight tests can't catch it.
  • Llama 3.2's rope_scaling factor is 32, not 8; the wrong value only hurts past about 80k tokens (meta-llama/llama-models #241).
  • GPT-2's c_attn is a Conv1D stored [in, out], the opposite of nn.Linear.
  • Gemma's RMSNorm multiplies by (1 + w).
  • rms_norm_eps is 1e-5 for Llama and SmolLM2, 1e-6 for Qwen3 and Gemma 3.
  • IO-5. These are Hugging Face config.json fields, fetched from the Hub on 2026-10-03:
Model Notable config
SmolLM2-135M d=576, 9 heads / 3 KV heads, tied, theta 1e5
Qwen2.5-0.5B QKV bias, theta 1e6
Llama-3.2-1B rope_scaling llama3, factor 32
Qwen3-0.6B head_dim 128 ≠ d/h, QK-norm
Gemma 3 270M head_dim 256, sliding window 512
  • IO-6. bf16 to fp32 is a 16-bit shift, so it's exact. fp16 to fp32 is also exact.
  • IO-7. NumPy's format spec covers the .npy header. allow_pickle code execution has existed since CVE-2019-6446, so never accept the object dtype.
  • IO-8. onnx/onnx (Apache-2.0). Protobuf caps a message at 2 GB, so larger models use external data (onnx #3273). External-data paths had a path-traversal CVE (CVE-2024-27318, fixed in onnx 1.16).
  • IO-9. The format spec is docs/gguf.md in ggml-org/ggml (MIT). Its parsers have a long CVE history:
  • TALOS-2024-1913 and TALOS-2024-1914 (2024-03);
  • CVE-2025-53630 (integer overflow in gguf_init_from_file_impl);
  • CVE-2026-27940 (a heap overflow);
  • six more posted to oss-security on 2026-05-15.
  • IO-10. Keras's safe_mode keeps being bypassed: CVE-2025-1550, then CVE-2025-8747 (which bypassed that fix), then CVE-2025-9905 (.h5 loading ignores safe_mode).
  • IO-11. TorchScript is deprecated as of PyTorch 2.10 (release blog); 2.11 recommends torch.export.

TRN: Training and fine-tuning

  • Overall falsifier: Shuttleworth et al. (arXiv 2601.22708, 2026-01) found that with a tuned learning rate, plain LoRA matches or beats most of its variants. If a variant doesn't beat tuned vanilla LoRA on our benchmark, don't ship it.
  • "LoRA Without Regret" (Thinking Machines, 2025-09-29):
  • Attention-only LoRA underperforms; apply it to all layers.
  • The best learning rate is about 10× full fine-tuning's.
  • LoRA tolerates large batches worse.
  • Rank 1 is enough for RL.
  • "LoRA learns less and forgets less" (Biderman et al., arXiv 2405.09673, 2024-05): LoRA is clearly worse than full fine-tuning on code and math, and it forgets less.
  • TRN-2. Loshchilov and Hutter, "Decoupled Weight Decay Regularization", arXiv 1711.05101. Test: decay is applied as w -= lr * wd * w, not added to the gradient. The check passes (#47).
  • TRN-3, TRN-4. Global-norm clipping and warmup-plus-cosine schedules are the defaults in every recipe cited above (for example "LoRA Without Regret"). Clip after accumulation and before the optimizer step. Done (#48, #49).
  • TRN-6. Full fine-tuning is the baseline in Biderman et al. and in "LoRA Without Regret". Adam in fp32 needs about 16 bytes per parameter plus activations: about 2.2 GB for 135M parameters, about 16 GB for 1B.
  • TRN-5. Hugging Face Trainer fixed this gradient-accumulation normalization bug in v4.46 (2024-10). Falsifier: 4×8 accumulated doesn't equal 1×32 on ragged batches. The check passes (#50).
  • TRN-7. Hu et al., "LoRA", arXiv 2106.09685 (2021-06).
  • Merging into a quantized base changes the outputs (huggingface/peft #2321), so merge in fp32.
  • Falsifier: merged and unmerged outputs differ by more than 1e-5.
  • TRN-8. Under the epsilon rule, LRP is linear in the contributions, so relevance splits exactly between W₀x and the adapter term. This is our own derivation and needs a numerical check. Gamma and z+ act on the positive part of the summed weight, so merged and unmerged models give different maps.
  • Related: Soligo et al., arXiv 2506.11618 (2025-06), read rank-1 LoRA adapters as interpretable directions.
  • TRN-9. Wu et al., "ReFT", arXiv 2404.03592 (2024-04).
  • It fails on arithmetic (GSM8K).
  • It fails badly at concept suppression compared with rank-1 steering vectors (arXiv 2505.20809, 2025-05).
  • TRN-10. Liu et al., "IA³", arXiv 2205.05638 (2022-05).
  • TRN-11.
  • Shuttleworth et al., "LoRA vs Full Fine-tuning: An Illusion of Equivalence", arXiv 2410.21228 (2024-10): LoRA introduces "intruder" singular vectors.
  • Minder et al., arXiv 2510.13900 (ICLR 2026): narrow fine-tunes leave readable traces in activation differences. Mixing pretraining data into the fine-tune removes the trace, so a null diff doesn't prove nothing changed.
  • TRN-12. Activation checkpointing conflicts with LRP's cached activations.
  • TRN-13.
  • rsLoRA: Kalajdzievski, arXiv 2312.03732 (2023-12).
  • LoRA+: Hayou et al., arXiv 2402.12354 (2024-02).
  • DoRA: Liu et al., arXiv 2402.09353 (2024-02).
  • TRN-14.
  • PiSSA: arXiv 2404.02948.
  • VeRA: arXiv 2310.11454.
  • AdaLoRA: arXiv 2303.10512.
  • GaLore: arXiv 2403.03507.
  • QLoRA: arXiv 2305.14314.
  • Prefix tuning: arXiv 2101.00190.
  • Prompt tuning: arXiv 2104.08691.
  • BitFit: arXiv 2106.10199. Most modern LLMs have no biases, so BitFit is nearly useless for them.
  • Houlsby adapters: arXiv 1902.00751.
  • bf16 updates round away small steps. Zamirai et al., arXiv 2010.06192. Keep fp32 master weights.

LLM: Running real language models

  • LLM-1. SmolLM2 needs grouped-query attention. Qwen3 and Gemma 3 need head_dim separate from d_model / num_heads.
  • pulsatrix's RoPEModule rotates adjacent pairs, while Hugging Face Llama-family checkpoints use "rotate half".
  • Fails when: the AttnLRP rules stop conserving relevance under a mask. Re-validate against LXT.
  • LLM-2. SmolLM2, Qwen3 and Gemma 3 set tie_word_embeddings. Without views, an untied copy of Gemma 3 270M's embedding would cost about 680 MB in fp32.
  • LLM-3. Moved to the TOK epic below.
  • LLM-4. Sampling must use the seeded RNG from FND-7 so generations are reproducible.
  • LLM-5. Falsifier: cached and uncached logits differ.
  • LLM-6. The threshold (fp32, max abs diff under 1e-3) follows from IO-4's layout pitfalls: random-weight tests miss them, real weights on real text don't.
  • LLM-7.
  • Achtibat et al., "AttnLRP", arXiv 2402.05602 (ICML 2024).
  • LXT 2.1 (2025-07-10) supports Gemma 3 and Qwen3, and marks Qwen3 experimental with attribution skewed toward the first token.
  • Falsifier: per-token relevance correlation with LXT is below 0.99 on Llama-family models.
  • LLM-8.
  • Tuned lens: Belrose et al., arXiv 2303.08112 (2023-03).
  • AtP*: Kramár et al., arXiv 2403.00745 (2024-03). Plain attribution patching misses effects through attention saturation.
  • LLM-9. Gemma 3 technical report, arXiv 2503.19786 (2025-03). Gemma Scope 2 (Google DeepMind, 2025-12) publishes SAEs and transcoders for every layer of Gemma 3 270M and 1B.

TOK: Tokenizers

  • TOK-1. The component split (normalizer, pre-tokenizer, model, post-processor, decoder) is the one Hugging Face tokenizers uses, so a tokenizer.json maps onto it one section at a time.
  • Offsets are measured in bytes of the UTF-8 input. Characters would need a second index, and every consumer here (VIZ-6a, LLM-7) slices UTF-8 strings.
  • Fails when: a normalizer changes length (NFC, ▁ replacement) and offsets point at the normalized text instead of the original. Hugging Face keeps an alignment map for this; we need one too.
  • TOK-2.
  • mlc-ai/tokenizers-cpp (Apache-2.0) exists but wraps the Rust tokenizers library, which pulls in a Rust toolchain.
  • A C++ port of the Hugging Face tokenizer found double-space mismatches in 4 of 15 million lines (wangkuiyi/huggingface-tokenizer-in-cxx).
  • SmolLM2 and Qwen2.5 store merges as "a b" strings. Qwen3, Llama 3.2 and Gemma 3 store them as ["a", "b"] pairs (newer tokenizers releases).
  • Llama 3.2 sets ignore_merges: a pre-token that is already in the vocabulary is used whole, without merging.
  • Qwen's only normalizer is NFC, which needs Unicode composition tables. They are generated, like the category table.
  • Split regexes seen (2026-10-05): GPT-2, Llama 3 (same as cl100k_base, also Phi-4), Qwen, o200k_base (gpt-oss, Mistral Nemo: case-aware with \p{Lu}\p{Lt}\p{Lm}\p{Lo}\p{M}) and DeepSeek-V3 (three Splits, including CJK ranges). Five families in a year argues for an engine over hand-written pre-tokenizers.
  • The engine only needs what these patterns use, and must match the backtracking, leftmost-first semantics of Oniguruma, which Hugging Face tokenizers uses by default (fancy-regex behind a feature flag).
  • Falsifier: id mismatches on the 10,000-line corpus.
  • TOK-3. Gemma 3's normalizer replaces spaces with ▁, so its pre-tokenizer split does nothing, and whole lines go to BPE as one piece. BPE has to be linear-ish in the piece length, not quadratic.
  • TOK-6. Training was out of scope for the first tokenizer ("a project-sized undertaking"). A byte-level BPE trainer with a priority queue over pair counts is a few hundred lines; the hard part is matching Hugging Face's tie-breaking if we want identical vocabularies, and we don't need that.
  • TOK-8. A .tiktoken file has no merge list: BPE merges the adjacent pair whose concatenation has the lowest rank. gpt-oss ships both tokenizer.json and the o200k encoding, which gives a cross-check between TOK-2 and TOK-8.
  • TOK-7. T5's precompiled charsmap is a serialized double-array trie of normalization rules (316 KB base64 for t5-small).

XAI: Question-driven framework

  • Four reasons to explain. Adadi and Berrada, "Peeking Inside the Black-Box", IEEE Access 6 (2018): justify, control, improve, discover. The exact wording comes from secondary summaries; the PDF section still needs reading.
  • Nine question categories. Liao, Gruen and Miller, "Questioning the AI", CHI 2020, arXiv 2001.02478. The categories were checked against the paper's Figure 1.
  • Designers report that "why not" questions are common and poorly served.
  • Question-to-method mapping. Liao et al., "Question-Driven Design Process for Explainable AI User Experiences", arXiv 2104.03483 (2021-04). It notes an "availability bias": practitioners default to LIME and SHAP.
  • Evaluation levels. Doshi-Velez and Kim, arXiv 1702.08608 (2017-02): application-grounded, human-grounded and functionally grounded evaluation. A library can only own the functionally grounded level.
  • XAI-1. The enums mirror Liao et al. (2020) and Adadi and Berrada (2018) exactly, so reports can be compared against the papers' question bank.
  • XAI-2, XAI-3. Falsifier: users ignore the planner and call LRP directly. That's why the planner is optional and never hides the direct APIs.
  • XAI-4.
  • Counterfactuals: Wachter et al., arXiv 1711.00399 (2017-11). DiCE: Mothilal et al., arXiv 1905.07697. On image models counterfactuals turn into adversarial examples (Freiesleben, arXiv 2009.05487).
  • Anchors: Ribeiro et al., AAAI 2018. Coverage can be tiny, and precision is estimated, not guaranteed.
  • TREPAN surrogates: Craven and Shavlik, NeurIPS 1996. Falsifier: fidelity under about 85% at a depth a person can read.
  • Pertinent negatives (CEM): Dhurandhar et al., NeurIPS 2018.
  • ProtoDash: Gurumoorthy et al., ICDM 2019.
  • XAI-5.
  • Quantus: Hedström et al., JMLR 2023, arXiv 2202.06861. It groups metrics into six families: faithfulness, robustness, localisation, complexity, randomisation and axiomatic.
  • ROAD: Rong et al., ICML 2022.
  • The model-parameter randomization test comes from Adebayo et al., NeurIPS 2018. Done (#53): a model-independent "explainer" scores a similarity of 1.0 at every layer, and gradient × input decorrelates.
  • XAI-6. See the cross-cutting section above. Done (#54).

CFS: Counterfactuals and sensitivity

  • CFS-1. Goldstein et al., "Peeking Inside the Black Box", arXiv 1309.6392 (ICE, c-ICE, d-ICE). Friedman's H-statistic builds on two-feature PDP. scikit-learn's partial_dependence(kind="both") returns PDP and ICE together.
  • CFS-2. Apley and Zhu, arXiv 1612.08468. Fails when: bins hold too few points; quantile bins keep counts even.
  • CFS-3. Occlusion: Zeiler and Fergus, arXiv 1311.2901. Captum's Occlusion and FeatureAblation are the references. Gradients measure an infinitesimal neighborhood; ±δ measures a finite one, and the two disagree exactly where the model saturates.
  • CFS-4. Morris, Technometrics 1991; Campolongo et al. 2007 (μ*). Saltelli et al. 2010 for the Sobol estimators. SALib implements both. Sobol indices assume independent inputs. Falsifier: indices that move by more than their confidence interval between seeds at the default sample size.
  • CFS-5. Wachter, Mittelstadt and Russell, arXiv 1711.00399. MAD weighting is from the same paper. On images, counterfactuals become adversarial examples (Freiesleben, arXiv 2009.05487); the report says so.
  • CFS-6. Growing spheres: Laugel et al., arXiv 1712.08443.
  • CFS-7. DiCE: Mothilal, Sharma and Tan, arXiv 1905.07697 (validity, proximity, sparsity, diversity). Plausibility as kNN distance follows Pawelczyk et al., "CARLA", arXiv 2108.00783.

INT: Embedding and representation analysis

  • INT-1. Huh et al., "Platonic Representation Hypothesis", arXiv 2405.07987 (2024-05), defines mutual-kNN alignment.
  • Fails when: a few points become everyone's neighbor in high dimensions (hubness).
  • INT-2. Kornblith et al., ICML 2019, arXiv 1905.00414.
  • Falsifier: CKA between a trained and a random-init network stays high. That would mean it measures the architecture, not the learning.
  • Fails when: outliers sway it, and it can be manipulated (Davari et al., arXiv 2210.16156).
  • INT-3. Hewitt and Liang, EMNLP 2019. MDL probing: Voita and Titov, EMNLP 2020.
  • INT-4.
  • Anisotropy: Ethayarajh, EMNLP 2019.
  • TwoNN: Facco et al., Scientific Reports 2017; applied to deep networks by Ansuini et al., NeurIPS 2019.
  • INT-6.
  • TCAV: Kim et al., ICML 2018, arXiv 1711.11279. Concept vectors are unstable across random negative sets.
  • CRP: Achtibat et al., Nature Machine Intelligence 2023, arXiv 2206.03208.
  • INT-7.
  • RSA: Kriegeskorte et al., 2008.
  • SVCCA: Raghu et al., NeurIPS 2017.
  • PWCCA: Morcos et al., NeurIPS 2018.
  • INT-8. t-SNE and UMAP distort global distances and cluster sizes (Wattenberg et al., Distill 2016; Chari and Pachter, PLoS Comp Bio 2023).

TDA: Topological data analysis

  • Core theory.
  • Persistent homology: Edelsbrunner, Letscher and Zomorodian, DCG 2002; Zomorodian and Carlsson, DCG 2005.
  • Stability: Cohen-Steiner, Edelsbrunner and Harer, DCG 2007.
  • Persistence modules as functors, and the interleaving distance: Chazal, de Silva, Glisse and Oudot, The Structure and Stability of Persistence Modules (2016).
  • Single linkage is the unique stable functorial clustering: Carlsson and Mémoli, JMLR 2010.
  • Mapper as a cosheaf: de Silva, Munch and Patel, DCG 2016.
  • TDA-2, TDA-3. Distances via ‖x‖² + ‖y‖² − 2xyᵀ suffer fp32 cancellation for near-duplicate points; clamp at zero and carry the error as a diagram bound (stability gives d_B ≤ ‖D − D′‖∞). H0 equals single-linkage clustering, which is the minimum spanning tree (Carlsson and Mémoli 2010).
  • TDA-4. Ripser: Bauer, JACT 2021, arXiv 1908.02518. MIT, about 1000 lines, no dependencies. Pulsatrix reimplements its algorithm instead of vendoring it. Test oracles that need no other library: the Rips complexes of points on a circle (Adamaszek and Adams, Pacific J. Math. 2017), and Betti numbers from boundary-matrix ranks.
  • giotto-ph is AGPLv3, so it can't be vendored.
  • GUDHI's CGAL-dependent modules are GPLv3.
  • Practical limits on CPU [I]: H1 up to about 2–5k points, H2 up to about 1k.
  • TDA-5.
  • Bottleneck distance: Kerber, Morozov and Nigmetov 2017.
  • Wasserstein stability: Skraba and Turner, arXiv 2006.16824.
  • TDA-6.
  • Persistence landscapes: Bubenik, JMLR 2015.
  • Persistence images: Adams et al., JMLR 2017.
  • Betti curves are not stable under the bottleneck distance.
  • TDA-7. The landmark subsampling error is at most twice the covering radius. Witness complexes: de Silva and Carlsson 2004.
  • TDA-8.
  • Bootstrap confidence sets: Fasy et al., Annals of Statistics 2014.
  • Permutation tests: Robinson and Turner 2017.
  • The universal null: Bobrowski and Skraba 2023 (Sci. Rep., arXiv 2207.03926). ℓ = A log log(death/birth) + B with B = −γ − A·L̄; A = 1 for Vietoris–Rips and 1/2 for Čech (not the other way round); null = left-skewed Gumbel, p = exp(−e^ℓ), Bonferroni. A conjecture; assumes a mostly-noise diagram.
  • TDA-9. Barcodes and persistence diagrams follow VIZ-1's JSON format; the significance band from TDA-8 is drawn on the diagram.
  • TDA-10.
  • Engels et al., "Not All Language Model Features Are Linear", ICLR 2025, arXiv 2405.14860: days of the week and months form circles in GPT-2 and Mistral.
  • Goodfire's BSF vision post (2026-07-07) measures block dimension by stable rank, a linear measure.
  • Circular coordinates: de Silva, Morozov and Vejdemo-Johansson, DCG 2011.
  • Falsifier: the dominant H1 bar isn't significant against the null, or it also appears on random-init activations.
  • TDA-11. RTD: Barannikov, Trofimov et al., ICML 2022 (arXiv 2201.00058). The practical 2n-vertex matrix is [[0, w₊ᵀ], [w₊, min(w, w̃)]] with w₊ = w with its upper triangle set to +∞; distances divided by their 0.9 quantiles; RTD₁ = total H1 bar length, averaged over both directions and batches of 500. Theorem 2.1 (empty for w = w̃; dimension k + 1 against all-zero w̃ equals the barcode of w in dimension k) makes a test oracle.
  • TDA-12. Naitzat, Zhitnikov and Lim, "Topology of Deep Neural Networks", JMLR 2020, arXiv 2004.06093.
  • TDA-13.
  • Distance concentration hides loops in high dimensions: Damrich, Berens and Kobak, NeurIPS 2024, arXiv 2311.03087.
  • Distance-to-measure: Chazal, Cohen-Steiner and Mérigot 2011.
  • TDA-14.
  • Mapper: Singh, Mémoli and Carlsson 2007.
  • Instability and parameter dependence: Belchí et al., JMLR 2020; Carrière, Michel and Oudot, JMLR 2018.
  • TDA-15. Carrière et al., ICML 2021. Topological autoencoders: Moor et al., ICML 2020.
  • TDA-16. TopoTroj: Zheng et al., NeurIPS 2021, arXiv 2106.06469.
  • Falsifier: a probe on norms or kNN distances matches it.
  • TDA-17 to TDA-19. Zigzag persistence on LLM layers: Gardinazzi et al., ICML 2025, arXiv 2410.11042. Multiparameter persistence has no complete invariant (Carlsson and Zomorodian 2009).
  • Neural persistence. Rieck et al., ICLR 2019. It mostly reduces to weight variance (Girrbach et al., arXiv 2307.10865, 2023).
  • Caution on recent LLM-topology papers. Most from 2025–26 use H0 only and are unreviewed preprints. H0 is a function of the minimum spanning tree, so check those results against plain kNN statistics.

KD: Surrogates and student-teacher

  • KD-1. Hinton, Vinyals and Dean, arXiv 1503.02531 (2015-03).
  • Students often don't match teachers even with enough capacity: Stanton et al., "Does Knowledge Distillation Really Work?", NeurIPS 2021.
  • KD-2. If prediction agreement is high but explanation agreement is low, that is the interesting signal, not a bug.
  • KD-3.
  • FitNets: arXiv 1412.6550.
  • Attention transfer: arXiv 1612.03928.
  • Soft decision trees: Frosst and Hinton, arXiv 1711.09784.
  • Boolean rule column generation: Dash, Günlük and Wei, NeurIPS 2018.
  • KD-4.
  • Born-again networks: Furlanello et al., arXiv 1805.04770.
  • DistilBERT: Sanh et al., arXiv 1910.01108.

FEAT: Featurizers

  • FEAT-6, BSF.
  • Fel et al., "Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds", arXiv 2606.25234 (2026-06-23).
  • The code is at github.com/goodfire-ai/block-sparse-featurizer. The variants are:
    • vanilla: keep the top-k blocks by norm;
    • Grassmannian: tied, with orthonormal block frames;
    • group lasso: a block soft-threshold.
  • The paper evaluates with minimum description length and stable rank.
  • The follow-up, Jerpelea and Ananthram, "A Deeper Analysis of Block-Sparse Featurizers", arXiv 2608.27515 (2026-08-27), reports that BSF still suffers from feature splitting and composition, and proposes tournament top-k.
  • Falsifier: on pulsatrix-scale models, BSF fails to beat a TopK SAE at matched L0.
  • FEAT-2. Gao et al., arXiv 2406.04093 (2024-06), introduces the auxiliary loss for dead latents.
  • FEAT-3.
  • SAEBench: Karvonen et al., arXiv 2503.09532 (2025-03). Its authors note that proxy metrics don't reliably predict usefulness.
  • Feature absorption: Chanin et al., arXiv 2409.14507.
  • FEAT-4.
  • BatchTopK: arXiv 2412.06410.
  • JumpReLU: arXiv 2407.14435.
  • Matryoshka: arXiv 2503.17547.
  • FEAT-5.
  • Transcoders: Dunefsky et al., arXiv 2406.11944.
  • "Transcoders Beat Sparse Autoencoders for Interpretability": Paulo et al., arXiv 2501.18823.
  • FEAT-7. AxBench (Wu et al., arXiv 2501.17148, 2025-01): SAE steering underperforms simple baselines. Steering vectors are unreliable in distribution and misgeneralize out of it (Tan et al., NeurIPS 2024).
  • FEAT-8. Crosscoders: Lindsey et al., Transformer Circuits (2024-10). Delta-Crosscoder: arXiv 2603.04426 (2026-03).
  • FEAT-9.
  • APD: arXiv 2501.14926.
  • SPD: Bushnaq, Braun and Sharkey, arXiv 2506.20790.
  • VPD: adversarial parameter decomposition. Bushnaq et al., "Interpreting Language Model Parameters", Goodfire technical report (2026-05-05), goodfire.ai/research/interpreting-lm-parameters. The report, the SPD and APD papers, and the code (SPD at tag v1, a single-file VPD in nano_param_decomp/run.py) are in github.com/goodfire-ai/param-decomp (formerly goodfire-ai/spd).
  • FEAT-10. Ameisen, Lindsey et al., "Circuit Tracing", Transformer Circuits (2025-03-27). The authors list limits: QK circuits aren't explained, there is reconstruction error, and inactive features are missed.
  • SAE limits.
  • Linear probes beat SAE probes out of distribution: DeepMind, "Negative Results for SAEs on Downstream Tasks" (2025-03-26). After this the team deprioritized SAE research.
  • SAEs trained on different seeds don't agree: Paulo and Belrose, arXiv 2501.16615.
  • Position SAEs as discovery tools: Peng et al., arXiv 2506.23845.

ARCH: New architectures

  • ARCH-1. B-cos: Böhle, Fritz and Schiele, CVPR 2022, arXiv 2205.10268. B-cos language models: arXiv 2502.12992.
  • ARCH-2. Koh et al., ICML 2020, arXiv 2007.04612. Concept leakage: Mahinpei et al., arXiv 2106.13314.
  • ARCH-3. Switch Transformer: arXiv 2101.03961. OLMoE router analysis: arXiv 2409.02060. Experts tend to specialize by token and syntax rather than domain.
  • ARCH-4. ProtoPNet: arXiv 1806.10574. Human and model similarity don't always match: Hoffmann et al., arXiv 2105.02968.
  • ARCH-5.
  • GNNExplainer: arXiv 1903.03894.
  • GNN-LRP: Schnake et al., TPAMI 2021.
  • Explainer benchmarks are unreliable: Faber et al. 2021.
  • ARCH-6. Dao and Gu, arXiv 2405.21060. No SSD LRP reference implementation was found.
  • ARCH-7. Beck et al., NeurIPS 2024, arXiv 2405.04517.
  • ARCH-8. KAN: arXiv 2404.19756. In "KAN or MLP: A Fairer Comparison", arXiv 2407.16674, KANs lose outside symbolic formulas.
  • ARCH-9. Peebles and Xie, arXiv 2212.09748. Moved to DIFF-6; see DIFF.
  • ARCH-10.
  • Gated DeltaNet: arXiv 2412.06464.
  • Titans: arXiv 2501.00663.
  • Hopfield layers: arXiv 2008.02217.
  • V-JEPA 2: arXiv 2506.09985.
  • SENN: arXiv 1806.07538.
  • Hypernetworks: arXiv 1609.09106.
  • Neural ODEs: arXiv 1806.07366.
  • Liquid time-constant networks: arXiv 2006.04439.

VIZ: Visualization pack

  • Tools go stale. CircuitsVis's last release was about a year ago, and token-hover regressions are still open (#98) [C]. Captum Insights was deprecated in Captum 0.8 (2025-03) [C].
  • VIZ-1.
  • Fails when: NaN and infinity appear in JSON (they're invalid), or the schema churns. Version it from day one and add golden-file tests. Done (#62): versioned from v1, non-finite numbers carried as null plus a JSON Pointer entry, and golden files byte-compared on Linux and Windows.
  • VIZ-2. ImPlot has no vector export, so publication figures need a separate renderer [I]. Share colormap.hpp so SVG and ImGui colors match. Done (#63): tests check the SVG fills against colormap.hpp exactly.
  • VIZ-3. Vega-Lite embedding docs. Inlining the JavaScript adds roughly 800 KB per file [I].
  • VIZ-4. circuit-tracer (Anthropic open-sourced it, 2025). Neuronpedia publishes an attribution-graph JSON schema and a validator at neuronpedia.org/graph/validator.
  • Falsifier: the schema needs transcoder-specific fields pulsatrix can't fill. Check against the validator first.
  • VIZ-6. Reference designs: SAEDashboard (jbloomAus), Neuronpedia feature pages, the TensorBoard projector, and SHAP's plots.
  • VIZ-7.
  • imnodes: maintained, but has no zoom.
  • imgui-node-editor: upstream is stale; the imgui_bundle fork is maintained (discussion #428, 2026-01) [C].

NB: Notebook layer

  • NB-1. xeus-cpp's xcpp::display calls mime_bundle_repr without qualification, so argument-dependent lookup finds an overload in the pulsatrix namespace. This was read from include/xcpp/xdisplay.hpp.
  • Falsifier: xcpp::display(tensor) in xeus-cpp 0.10 doesn't pick up the overload.
  • NB-2. The nbformat v4 format description: since 4.5, every cell needs an id. MIME types ending in +json are stored as objects, and PNG data is base64.
  • Falsifier: the output fails nbformat.validate, or doesn't render on GitHub.
  • NB-3. anywidget (MIT, 0.11.0, 2026-04-27) works in Jupyter, JupyterLab, Colab, VS Code and marimo.
  • NB-4. xeus-cpp 0.10.0 (BSD-3-Clause, released 2026-04-02).
  • Kernel crashes: a segfault or failed assert restarts the kernel (#434), and lambdas with xwidgets crash it (#476).
  • GPU: CUDA math functions return 0 inside JIT-compiled kernels (#467).
  • Windows: "Does not work on Windows" is still open (#144).
  • xeus-cling hasn't been updated since 2025-10, so don't target it.
  • NB-5. xeus-cpp-lite (Jupyter blog, 2025-06-19) loads libraries as emscripten-forge side modules. Threads need SharedArrayBuffer headers [I].

HIP: Training efficiency on AMD GPUs

  • Hardware ceiling.
  • Memory bandwidth: 256 GB/s theoretical (LPDDR5X-8000, 256-bit), about 212 GB/s measured on the GPU [C] (llm-tracker Strix Halo page).
  • fp16/bf16 peak: about 59 TFLOPS from WMMA [V] (GPUOpen, "WMMA on RDNA3"). About 37 TFLOPS bf16 measured with an optimized stack [C].
  • fp32 peak: about 15 TFLOPS without dual issue [I].
  • So nearly every non-GEMM kernel is bandwidth-bound or worse [I].
  • warpSize and shuffles. In ROCm 7.0, warpSize is no longer constexpr, and *_sync shuffles take 64-bit masks [V] (ROCm 7.0 release notes; ROCm/HIP #3667).
  • gfx1151 ISA. The LLVM AMDGPU target table lists gfx1151 as an APU with no XNACK [V] (LLVM AMDGPUUsage).
  • HIP-1.
  • rocprofv3 (ROCprofiler-SDK) is the standard profiler in ROCm 10.0 [V].
  • rocprof-compute support on gfx115x landed in TheRock PR #8299 [C]. ROCm 10.0 fixed its roofline precision list [V]. It's unverified on 7.2.4.
  • HIP-2.
  • Falsifier: rocprofv3 shows these kernels take under 5% of step time on real workloads. It held for softmax, the norms and column_sums, which were left alone. It failed for BatchNorm (79% of the CNN's kernel time), which was rewritten: step time fell from about 27 ms to about 10 ms (#57).
  • Fails when:
    • a 32-bit mask truncates the shuffle on 64-wide waves;
    • results differ from the CPU in the last bit (use tolerances);
    • a wave64 build changes the reduction order.
  • HIP-3.
  • Design: PyTorch's caching allocator, described by Zach DeVito (2022) and in the PyTorch devlog on fragmentation (2026-06-01).
  • Avoid hipMallocAsync on RDNA [C]:
    • silent corruption on gfx1030 (CTranslate2 #2090);
    • OOM reported while memory is still consumed (rocm-systems #11642);
    • double free on pool failure (rocm-systems PR #11643).
  • VRAM on the APU [C]:
    • ROCm adds VRAM and GTT together when reporting capacity, which overstates what can be allocated (ROCm #6004).
    • AMD recommends a 0.5 GB carve-out on Strix Halo [V] (AMD Strix Halo system optimization guide).
  • HIP-4. Falsifier: the GPU is already more than 90% busy with the syncs in place. It didn't hold: HIP-1 measured 12–13% busy on the small models, and removing the syncs made them about 2× faster (#59).
  • Fails when: the host reads stale data, or an error surfaces in a later op.
  • HIP-5. dot and sum launch a single 256-thread block today (src/gpu_kernels.cuh). Use per-block partials and a second pass; no atomics, to keep results deterministic. Done (#56): both now run at the measured memory bandwidth.
  • HIP-6. Fails when: the fused training path and the unfused explain path drift apart numerically. Gate them with a conservation test.
  • HIP-7. At N=64, a 3×3 kernel on 224×224 input needs about 350 MB of im2col buffer [I].
  • MIOpen on gfx1151 has a history of trouble [C]: missing Composable Kernel libraries (TheRock #5105, closed), and a Winograd lockup (TheRock #5581).
  • HIP-8. Sources disagree [C].
  • Against: ROCm #5643 (ROCm 7.1.0) reports "unsupported architecture".
  • For: the changelog and community guides say ROCm 7.2.4 ships gfx1151 kernels.
  • An older run measured hipBLASLt slower than rocBLAS.
  • A 30-line probe settles it.
  • HIP-9.
  • ROCm 10.0.0 (2026-08-26) is the first release whose notes list the Ryzen AI Max parts as gfx1151 [V] (release notes, compatibility matrix).
  • ROCm 7.2 page faults on host-to-device copies (ROCm #5890, open) [C].
  • A wrong VGPR count crashed gfx1151 through ROCm 7.1.x. The user-mode fix landed in 7.2; the kernel-mode fix needs Ubuntu's OEM kernel 1018 or newer (TheRock #2991) [C].
  • AMD asks for host kernel 6.18.4 or newer on other distributions [V].
  • Done (#65): on gfx1151 with host kernel 7.0.0, ROCm 10.0.0 passed every HIP test and matched 7.2.4 within 2%; it is now the default. ROCm #5890 didn't show up on either version.
  • HIP-10. HIP's unified-memory documentation [V]: without XNACK, hipMallocManaged and hipHostMalloc give pinned, zero-copy host memory.
  • HIP-11. LRP's stabilized divisions and conservation checks degrade in bf16, which has an 8-bit mantissa [I]. That's why the rule is to explain in fp32.
  • HIP-12. Reports that this work may not pay off [C]:
  • hipGraph is 2–5% slower than eager execution on gfx11 (legacy-rocm-build #6685);
  • HIP graph branches get serialized on gfx1201 (rocm-systems #12330);
  • Composable Kernel's flash attention on RDNA3 is forward-only (CK #1434);
  • rocWMMA flash attention regresses prefill by 41% on gfx1151 (llama.cpp #24437).

AGT: Agents

  • No official C++ SDKs.
  • Anthropic's official SDKs are Python, TypeScript, C#, Go, Java, PHP and Ruby [V] (client SDK docs, checked 2026-10-03).
  • MCP's official SDK tiers [V] (modelcontextprotocol.io/docs/sdk):
    • Tier 1: TypeScript, Python, C#, Go, Rust and Ruby.
    • Tier 2: Java.
    • Tier 3: Swift, PHP and Kotlin.
  • Community C++ MCP libraries.
  • gopher-mcp (Apache-2.0, v0.1.18): its own issue #294 says 2026-07-28 support had only been tested against itself.
  • cpp-mcp (MIT): targets older spec revisions.
  • Neumann-Labs/mcp-cpp: GPL-3.0, so it can't go into an MIT library.
  • AGT-1. nlohmann/json (MIT, v3.12.0). xeus-cpp's display API also uses it, so one dependency serves AGT, VIZ-1 and NB-1. Throughput doesn't matter at these payload sizes.
  • AGT-2.
  • cpp-httplib (MIT, v0.59.0): requires OpenSSL 3.0 or newer. Issue #2596 (opened 2026-09-29) reports Windows certificate verification failures.
  • cpr (MIT-style) wraps libcurl, which can use Windows' own TLS stack.
  • Falsifier: Windows CI fails to verify api.anthropic.com. Then switch the default to libcurl.
  • AGT-3: Anthropic API behavior [V] (define-tools, handle-tool-calls, rate-limits and streaming docs, 2026-10-03):
  • tool_choice: any and tool_choice: tool return HTTP 400 on the current 5.x models.
  • tool_result blocks must come first in the next user message.
  • A 429 from a spend limit has error code enforced_spend_limit_reached and no retry-after header.
  • New stream event types can appear at any time.
  • Streamed tool input arrives as input_json_delta fragments, parsed at content_block_stop.
  • AGT-4.
  • MCP schemas default to JSON Schema 2020-12, but the main C++ validator (pboettch/json-schema-validator 2.4.0) supports draft-7 only.
  • Falsifier: more than two real tools need $ref or oneOf.
  • AGT-5.
  • MCP 2026-07-28 changelog [V]: no initialize handshake, no sessions, and server/discover is required.
  • stdio transport [V]: nothing but protocol messages on stdout, and no embedded newlines.
  • tools spec [V]: servers must validate inputs, apply access controls, rate-limit and sanitize output.
  • Security statistics about popular MCP servers (for example, 43% with command injection) come from secondary vendor reports (Checkmarx), not primary scans.
  • AGT-6.
  • Grammar-constrained decoding is fragile at the edges of the schema language (llama.cpp #25923, #22314, #29006).
  • XGrammar (Apache-2.0, v0.2.8) has a C++ core.
  • SmolLM2-135M reaches about 22% exact match on xLAM function calling even after full fine-tuning (chloe-mp/smollm2-xlam-function-calling) [C]. Grammar constraints guarantee well-formed calls, not correct ones.
  • AGT-7. No prior art or benchmark was found for attributing an agent's tool choice. The falsifier is the deletion test in the roadmap.
  • AGT-8.
  • "Language Models Can Explain Neurons": Bills et al., OpenAI (2023-05).
  • Delphi: Paulo et al., arXiv 2410.13928 (2024-10). Its explanations fail about 38% of the time (arXiv 2605.01555, 2026-05).
  • Output-centric descriptions: arXiv 2501.08319.
  • AGT-9. MAIA: Shaham et al., arXiv 2404.14394 (ICML 2024). Transluce's Docent (Apache-2.0) analyzes agent transcripts.
  • Prior art. mozilla-ai/agent.cpp (MIT, 2025-12) is a C++17 agent loop tied to llama.cpp, with no MCP. It's a design reference, not a dependency.

KS: Kitchen sink

  • KS-1. The README said "There's no install() step yet." Done (#61).
  • KS-2. The benchmark suite provides the numbers HIP-1 through HIP-8 are judged by, plus conservation error so performance work can't silently break LRP. Done (#64): a 1000× larger LRP stabilizer fails the conservation gate (0.13 against 10⁻³), and restoring HIP-4's syncs is flagged as a regression.
  • KS-3.
  • Monte Carlo dropout: Gal and Ghahramani 2016.
  • Deep ensembles: Lakshminarayanan et al. 2017.
  • Split conformal prediction: Vovk et al.; Angelopoulos and Bates, arXiv 2107.07511.
  • KS-4.
  • FGSM: Goodfellow et al., arXiv 1412.6572.
  • PGD: Madry et al., arXiv 1706.06083.
  • KS-5. TracIn: Pruthi et al., NeurIPS 2020. Captum 0.8 added influence functions [C]. Kronfluence's EK-FAC covers only linear and conv layers [C].
  • KS-6. docs/data-pipeline/index.md currently calls drift detection and fairness metrics out of scope. Model cards: Mitchell et al., FAT* 2019.
  • KS-7. DataThreadPool and BoundedQueue exist but aren't wired into DataLoader (data_loader.hpp).
  • KS-8. The HIP backend is tested on real gfx1151 hardware but not in CI (README Status).
  • Risk: runner flakiness from the gfx1151 driver issues listed under HIP-9.
  • KS-9. Llama 3.2 is under the Llama 3.2 Community License. Ship converters, not converted weights.
  • KS-11. Python wheels with GPU backends are the hard part. A CPU-only wheel and a vcpkg port (mlpack and dlib both ship one) come first.
  • KS-10, KS-12. These close gaps against the other libraries we compared:
Library What it has that pulsatrix lacks
PyTorch, JAX Views, broadcasting, op-level autograd
tinygrad Automatic fusion
ggml and llama.cpp GGUF and quantization
Captum Influence functions
Quantus Evaluation metrics
TransformerLens and nnsight Hook points
SAELens SAE training at scale

PLM: Protein language models

The full write-up is in Protein language models: research and plan.

  • Choice of model. ESM-2 (Lin et al., Science 2023). The facebook/esm2_* checkpoints ship model.safetensors, config.json and vocab.txt under MIT [V] (Hugging Face, checked 2026-10-07). ESM C's license depends on size; check each one before supporting it.
  • PLM-1, PLM-2. The Hugging Face EsmConfig sets position_embedding_type ("rotary" for ESM-2), token_dropout, emb_layer_norm_before and layer_norm_eps [V].
  • Falsifier: logits that don't match transformers to float precision on esm2_t6_8M.
  • Fails when: token-dropout scaling is skipped at inference. The input embeddings are then off by the mask-ratio factor even with no masks.
  • Result (PLM-2, #95): the falsifier doesn't fire. 8M and 650M match within 4e-6 relative (logits, hidden states) and 1.3e-5 absolute (attention), once RoPE uses the checkpoint's stored fp16-rounded inv_freq [V]. Computing it exactly misses by 2e-4.
  • PLM-3. ProteinGym: 217 DMS substitution assays, scored by masked-marginal log-odds and Spearman correlation [V]. The best methods reach about 0.45–0.48 average Spearman (MSA Pairformer, VespaG) [V].
  • Falsifier: per-assay Spearman more than 0.01 from ProteinGym's published ESM-2 scores.
  • PLM-4. Contacts: ESM's logistic regression over attention maps after symmetrizing and APC. arXiv 2606.21876 (2026): averaging the top-K heads, chosen with 10 labeled proteins, matches the Categorical Jacobian in one forward pass [V].
  • Fails when: testing in distribution. Leakage-clean splits score 30–36 points lower [V].
  • Result (PLM-4): contact probabilities match transformers within 3.1e-5 on ESM-2 8M and 650M [V]. With 650M, on five well-known structures, the ten best heads (ranked on five others) score a mean long-range P@L of 0.642, against 0.637 for the trained head [V]. These proteins are in distribution, so the numbers are optimistic.
  • PLM-6. arXiv 2606.22181 (2026): integrated gradients on a well-performing ESM-2 allergen classifier didn't recover annotated epitopes [V]. Residue explanations need quantitative checks against an independent signal, not just a plausible look.
  • Result (PLM-6): AttnLRP on ESM-2 650M matches LXT's rules to 4.8e-5 [V]. A per-protein relevance profile agrees with DMS sensitivity and with conservation at Spearman 0.03-0.10 on ubiquitin and TEM-1, where conservation and DMS agree at 0.29-0.50 [V]: the same warning, from AttnLRP. Deletion confirms faithfulness on both, but TEM-1's explanation survives randomizing the top 14 of 33 layers.
  • PLM-7. ESM-2 sampled UniRef50 clusters uniformly, with a random UniRef90 member per cluster; 15% of tokens are masked 80/10/10 [V] (NVIDIA BioNeMo docs). UniRef-only masked-LM training overfits, which metagenomic data mitigates (Cheng et al., ICML 2024 workshop) [V]. Yearly UniRef snapshots show no data saturation yet (arXiv 2507.22210) [V].
  • Result (PLM-7): a training step matches transformers. BioNeMo publishes no 8M curve, only 650M and 3B validation perplexity (7.00 and 6.00, on a 2024 UniRef50 sample) [V]. On our UniRef50 sample, the published checkpoints score 10.89 (8M) and 5.96 (650M) [V].
  • PLM-8. InterPLM (Simon and Zou, Nature Methods 2025): SAEs with 10,420 latents on ESM-2-8M (all 6 layers) and 650M, matched to Swiss-Prot concepts. Neurons are much less interpretable than SAE features [V].

DIFF: Diffusion models

The rules these notes support are on the design page. Researched 2026-10-20.

  • DIFF-1, novelty. No published LRP, DTD, DeepLIFT or conservation-based attribution was found for DDPM/DDIM samplers, U-Nets, DiT, adaLN/FiLM, flow matching or diffusion LMs. The search covered about eight targeted web and arXiv queries and the zennit, zennit-crp and LXT issue trackers. The AttnLRP author, replying in LXT issue #36 (2025-08-13): "I haven't yet applied LRP to a diffusion model" [V]. Captum's issues, GitHub Discussions and workshop papers weren't searched, so treat novelty as likely, not certain [I].
  • Falsifier: a paper or LXT release with LRP through a sampler.
  • Fails when: issue #36's user seeded gradient × input on the noise prediction. Relevance sign followed the predicted noise's sign, and maps changed from run to run [V]. That's why D1 seeds a scalar of x_0.
  • Nearest neighbours, all one step at a time.
  • DAAM (Tang et al., arXiv 2210.04885, 2022-10): cross-attention summed over layers, heads and timesteps. Noun segmentation mIoU80 64.8 on SD 2.0 [V]. Its authors called gradient attribution through the sampler intractable, because small perturbations gave very different images [V].
  • Kong et al. (arXiv 2310.07972, 2023-10): conditional mutual information heatmaps. They track image change under word swaps better than attention (Pearson 0.34 against 0.24) [V].
  • DifFRACT (Mazur, Konovalova and Alanov, arXiv 2606.15796, 2026-06): attribution graphs inside one FLUX.1-schnell step, made exact by freezing LayerNorm denominators and attention probabilities and substituting timestep-conditioned transcoders. The text stream's share of attribution falls from 89.9% at the first step to 5.4% at the last [V]. It never chains steps.
  • DLIG (Aswal and Hudelot, arXiv 2610.01177, 2026-10): Integrated Gradients on layer activations at one partially unmasked state, with committed tokens held fixed in the baseline. Tested only on DiffuGPT-M (355M). On WiC, commit timing separated Yes from No (AUC 0.342) where attribution depth didn't (0.503). On infilling, the model's own tokens as target correlate with quality at r = 0.013, a fixed reference at 0.197 [V].
  • Attribution contract (arXiv 2605.23080, 2026-05): defines state-level attribution for diffusion LMs without chaining it. In autoregressive models 29% of IG mass lands on the model's own generated prefix [V].
  • DLM-Scope (arXiv 2602.05859, 2026-02): Top-K SAEs on Dream-7B and LLaDA-8B, separate for masked and unmasked positions. Inserting them in early layers lowers cross-entropy [V].
  • TimeROME-DLM (arXiv 2606.12841, 2026-06): causal tracing across steps on LLaDA-8B [V, abstract only].
  • Corrections: arXiv 2606.12273 is AGDO, an attention-guided unmasking order for post-training, not a study of attention across steps. arXiv 2610.18407 proposes a multi-stream residual architecture, not a mechanism study of existing models [V].
  • D1, the split. Otsuki et al., ECCV 2024, arXiv 2407.09115: relevance at a residual merge split by absolute value, R_s = R·|h_s| / (|h_m| + |h_s|). Exactly conservative and bounded [V]. Arras et al., arXiv 1909.12114 (2019-09), LRP through time for LSTMs: the precedent for unrolling a recurrence. Their signed proportional split needed ε = 0.2 to be stable and scored near random [V].
  • Falsifier: conditioning relevance doesn't concentrate in early steps. Critical windows (Li and Chen, arXiv 2403.01633, 2024-03) and phase transitions (Sclocchi et al., arXiv 2402.16991, 2024-02) fix class and layout early [V, abstracts].
  • Fails when: the skip and network terms cancel, so the split reports magnitude, not contribution [I]. Randomizing the network can't change the skip share, so a randomization test that "passes" may only reflect the skip path [I].
  • DIFF-5 result: on a planted class-conditional DDPM the absolute-value split gave the class the wrong sign in 9 of 12 samples; a sign-keeping bounded ratio, R h_k sign(Σh) / Σ|h|, tracked a per-step interventional ground truth (median Spearman 0.98) and is the default [V, this repo].
  • D1, the seed. Kobayashi et al., arXiv 2303.11734 (2023-03), LRP for autoencoders: seed the scalar reconstruction error, rooted at the input. Per output unit (1/m)(x̃_i − x̂_i)² under L2, exactly conservative [V].
  • D2, commit as identity. LLaDA's generate.py gives already committed positions confidence −∞, so they're never revised. MDLM copies unmasked tokens forward [V]. Neither carries hidden state between steps, so commits are the only channel across steps [I]. The attribution-contract paper: with the unmasking order replayed, IG's completeness residual on a masked diffusion model falls from 97% at 1 step to 18% at 256 steps [V].
  • Falsifier: prompt relevance disagrees with IG under the same frozen commit order, or prompt deletion doesn't beat random.
  • Fails when: the sampler re-masks. DPLM's default _reparam_decoding re-masks the lowest-scoring positions, including decoded ones, every step [V], and so does ReMDM (arXiv 2503.00307) [V]. In DPLM, decoded positions that survive keep their old token, and only masked ones take the new prediction (it starts from the previous step's tokens; DIFF-4 matches its generate step by step) [V]. Hence "last write" instead of "commit". KV or feature caching (Fast-dLLM, dKV-cache) adds a channel the graph misses [I].
  • Mask sinks: mask tokens are 0.366 of attention-sink tokens in LLaDA and 0.321 in Dream. Masking the top sink moves LLaDA's GSM8K from 0.76 to 0.75. The paper claims sink removal costs under 1%, but its own Dream score falls from 0.82 to 0.75 (Rulli et al., arXiv 2510.15731, 2025-12) [V].
  • Sampler dependence: Dream scores 8% on GSM8K under random commit order against 56–59% under margin or entropy order (DLM-Scope) [V]. A graph is only valid for the sampler it recorded.
  • D3, split or detach. AttnLRP (arXiv 2402.05602): the uniform rule gives each factor of a product an equal share, assuming the product vanishes when the factors do [V]. MambaLRP (arXiv 2406.07592): splitting a gate half and half beats detaching one factor, 1.978 against 1.577 on SST-2 and 4.715 against 3.592 on ImageNet [V]. Arras et al.: for LSTM gates, giving the gate nothing ("signal-take-all") beats every split, which scored near random [V]. No LRP rule for FiLM or conditional normalization was found.
  • Falsifier: split mode fails a planted class-controls-one-region DiT.
  • Fails when: 1+γ is near zero, so the LN(x) and LN(x)·γ terms cancel [I].
  • DIFF-2 to DIFF-4, checkpoints (read from configs and safetensors headers, 2026-10-20).
  • kuleshov-group/mdlm-owt: 169,627,250 parameters, fp32 safetensors, Apache-2.0. A 12-block DiT, hidden 768, 12 heads, RoPE (base 10000), GPT-2 BPE plus a mask token (vocab 50258). time_conditioning: false forces σ = 0, so the adaLN vectors are constants [V]. Its remote code hard-imports flash_attn [V]. Its rotary layout (interleaved or rotate-half) wasn't checked.
  • airkingbd/dplm_150m and dplm_650m: state-dict keys and shapes identical to EsmForMaskedLM, pytorch_model.bin only, token_dropout: true, no timestep input, no license tag on the cards [V]. ESM-2 initialization isn't confirmed [C].
  • airkingbd/dplm2_*: vocab 8229 with structure tokens, split RoPE over [struct ; aa]. The <mask_struct> id differs between tokenizer_config.json (8229) and vocab.txt (8228) [V].
  • louaaron/sedd-small: the same DiT keys as MDLM-owt, but with real σ conditioning, so adaLN can't be folded away [V].
  • dllm-hub/Qwen3-0.6B-diffusion-mdlm-v0.1: 0.75B, Apache-2.0, stock Qwen3 with a bidirectional mask [V]. Mask token <|mask|>, id 151669; no logit shift (position i predicts token i); its card samples LLaDA-style, by low confidence in blocks of 64 [V]. Its modeling code needs transformers 4.57 [V].
  • diffusionfamily/diffugpt-m: 355M, fp16, GPT-2 layout [V].
  • LLaDA-8B (8,015,581,184 parameters, MIT) and Dream-7B (7,615,616,512, Apache-2.0) need 32.1 GB and 30.5 GB for fp32 weights alone [V], beyond this machine.
  • DIFF-7 to DIFF-12, the toolkit. Logit lens: a single pass recovers most masked words (Devoto blog, about 2025) [C]. Prophet (arXiv 2508.19982, 2025-08): answers converge internally well before commit, up to 97% of GSM8K correct at half the steps [C]. Surkov et al. (arXiv 2410.22366, 2024-10): SAEs on SDXL-Turbo transfer from 1 to many steps [V, abstract]. Data attribution (TRAK, D-TRAK, K-FAC influence, arXiv 2410.13850) answers "which training images" and is out of scope [V, abstract].