|
pulsatrix 1.1.0
|
Tools for inspecting and probing a model's internal computation directly – activation caching, linear probes and their controls, representation similarity (linear CKA, RSA, SVCCA), nearest neighbors and mutual-kNN alignment, representation geometry (anisotropy, effective rank, intrinsic dimension), PCA projections, export for UMAP and t-SNE, TCAV and CRP, sparse autoencoders, circuit graphs – plus the GFlowNet-based non-goal-directed training objectives motivated by the Scientist AI research direction. More...
Files | |
| file | activation_snapshot.hpp |
| Self-contained, enumerable copy of one forward pass's cached activations. | |
| file | block_sparse_featurizer.hpp |
| Block-sparse featurizers (FEAT-6; Fel et al., "Structuring Sparsity: Block-Sparse
Featurizers Capture Visual Concept Manifolds", arXiv 2606.25234): features are small subspaces (blocks of directions) instead of single directions, so a concept that lives on a low-dimensional manifold, such as an angle or a lighting direction, is one feature. | |
| file | circuit_graph.hpp |
| Self-contained circuit-graph artifact – scored nodes and weighted edges. | |
| file | circuit_tracing.hpp |
| Attribution graphs with cross-layer transcoders (FEAT-10), after Ameisen, Lindsey et al., "Circuit Tracing: Revealing Computational Graphs in Language Models" (Transformer Circuits, 2025) and its open implementation, circuit-tracer (github.com/decoderesearch/circuit-tracer). Each MLP is replaced by a transcoder's features plus an error term, attention patterns and normalization scales are frozen, and what is left is linear: the graph's edges are exact attributions between features, errors, token embeddings and output logits. | |
| file | concept_relevance.hpp |
| Concept Relevance Propagation (INT-6): which of a layer's channels an LRP explanation passes through, where each one looks in the input, and which inputs show it at work. | |
| file | crosscoder.hpp |
| Crosscoders (FEAT-8): one sparse dictionary read from and written to several sources at once, such as the residual stream at several layers, or the same layer in a base model and its fine-tune (Lindsey et al., "Sparse Crosscoders for Cross-Layer Features and
Model Diffing", Transformer Circuits, 2024). Model diffing reads which latents' decoders live in one model only, with Minder et al.'s BatchTopK crosscoder and Latent Scaling ("Overcoming Sparsity Artifacts in Crosscoders to Interpret Chat-Tuning", arXiv 2504.02922), and Kassem et al.'s Delta-Crosscoder for narrow fine-tunes ("Delta-
Crosscoder: Robust Crosscoder Model Diffing in Narrow Fine-Tuning Regimes", arXiv 2603.04426). | |
| file | detailed_balance_loss.hpp |
| GFlowNet Detailed Balance loss (Bengio et al., "GFlowNet Foundations", arXiv:2111.09266). | |
| file | embedding_export.hpp |
| Hand embeddings or activations to UMAP and t-SNE tools, and bring their coordinates back as a projection document. | |
| file | featurizer.hpp |
| The featurizer family's common interface (FEAT-1): models that rewrite activations as sparse codes over learned directions and back, such as sparse autoencoders, with a shared training step and metrics (L0, dead and dense latents). | |
| file | featurizer_metrics.hpp |
| How good a featurizer is (FEAT-3), after SAEBench (Karvonen et al., arXiv 2503.09532): reconstruction (explained variance, cosine, norm ratio, L0, dead and dense latents), loss recovered when the reconstruction is spliced back into the model, and feature absorption (Chanin et al., arXiv 2409.14507) with a linear probe as its baseline. | |
| file | gflownet_forward_policy.hpp |
| Masked stochastic categorical policy over a logit-producing Module – P_F for GFlowNet training objectives. | |
| file | gflownet_trajectory.hpp |
| Rolls out one full HyperGrid episode under a GFlowNetForwardPolicy. | |
| file | hypergrid_env.hpp |
| HyperGrid – the standard minimal GFlowNet correctness-check environment. | |
| file | jumprelu_sparse_autoencoder.hpp |
| The JumpReLU sparse autoencoder (FEAT-4; Rajamanoharan et al., "Jumping Ahead:
Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders", arXiv 2407.14435): a learned threshold per latent, trained against an L0 penalty through straight-through estimators. | |
| file | learnable_scalar.hpp |
| A single trainable float, updated by plain SGD – not a Tensor/Module parameter. | |
| file | linear_probe.hpp |
| Linear probe – trains one linear classifier to decode a binary concept from a layer's activations, answering "is this concept linearly represented here?". | |
| file | mlp_hook.hpp |
| Reads or replaces a transformer block's MLP output during forward() (FEAT-5), for collecting a transcoder's training pairs and splicing it in. | |
| file | nearest_neighbors.hpp |
| Nearest neighbors over embeddings or activations, mutual nearest neighbors, and mutual-kNN alignment between two representations. | |
| file | parameter_decomposition.hpp |
| Parameter decomposition (FEAT-9): explaining a network in weight space. Each weight matrix is split into rank-one subcomponents, and training asks that, for every input, few of them be needed to compute the output. SPD (Bushnaq, Braun and Sharkey, "Stochastic Parameter Decomposition", arXiv 2506.20790) and VPD (Bushnaq et al., "Interpreting Language Model Parameters", Goodfire, 2026: adversarial parameter decomposition) are both here, after their reference code (github.com/goodfire-ai/ param-decomp: tag v1 for SPD, nano_param_decomp for VPD). | |
| file | pca.hpp |
| Principal component analysis of a layer's representations (INT-5): 2D and 3D views, directions to steer along, and fewer dimensions for what comes after. | |
| file | probe_controls.hpp |
| Linear probes with controls (INT-3): a multi-class probe trainer, Hewitt and Liang's control task and selectivity, and a comparison against a baseline representation. | |
| file | relevance_graph.hpp |
| A language model's attribution graph from AttnLRP (VIZ-4): how relevance for one predicted token flows from the input tokens through every layer's residual stream, ready for Neuronpedia's and circuit-tracer's graph viewers. | |
| file | representation_geometry.hpp |
| One number per layer for the shape of its representations: anisotropy, effective rank and TwoNN intrinsic dimension. | |
| file | representation_similarity.hpp |
| How similar two layers' representations of the same inputs are: linear CKA, RSA and SVCCA. | |
| file | sparse_autoencoder.hpp |
| Sparse autoencoder – reconstructs an activation through an overcomplete, L1-penalized hidden layer, decomposing it into a larger, sparser basis. | |
| file | steering.hpp |
| Steering a model by adding a direction to its residual stream (FEAT-7), with difference-of-means as the default direction, a featurizer's decoder direction as an option, and a report on how reliably steering works. | |
| file | subtb_loss.hpp |
| GFlowNet SubTB(lambda) loss (Madan et al., "Learning GFlowNets from partial
episodes for improved convergence and stability", arXiv:2209.12782). | |
| file | tcav.hpp |
| Testing with Concept Activation Vectors (INT-6): how much a concept, given only by examples, pushes a model towards a class, and whether that's more than chance. | |
| file | topk_sparse_autoencoder.hpp |
| The TopK sparse autoencoder (FEAT-2; Gao et al., "Scaling and evaluating sparse
autoencoders", arXiv 2406.04093): sparsity set directly by keeping each input's k largest latents, with an auxiliary loss that revives dead latents. Two variants (FEAT-4): BatchTopK (Bussmann et al., arXiv 2412.06410) and Matryoshka (Bussmann et al., arXiv 2503.17547). | |
| file | trajectory_balance_loss.hpp |
| GFlowNet Trajectory Balance loss (Malkin et al. 2022, arXiv:2201.13259). | |
| file | transcoder.hpp |
| Transcoders and skip transcoders (FEAT-5): sparse, wide replacements for an MLP that predict its output from its input (Dunefsky et al., "Transcoders Find Interpretable LLM
Feature Circuits", arXiv 2406.11944), optionally beside a linear skip connection (Paulo et al., "Transcoders Beat Sparse Autoencoders for Interpretability", arXiv 2501.18823). | |
Tools for inspecting and probing a model's internal computation directly – activation caching, linear probes and their controls, representation similarity (linear CKA, RSA, SVCCA), nearest neighbors and mutual-kNN alignment, representation geometry (anisotropy, effective rank, intrinsic dimension), PCA projections, export for UMAP and t-SNE, TCAV and CRP, sparse autoencoders, circuit graphs – plus the GFlowNet-based non-goal-directed training objectives motivated by the Scientist AI research direction.