Parameter decomposition (FEAT-9): explaining a network in weight space. Each weight matrix is split into rank-one subcomponents, and training asks that, for every input, few of them be needed to compute the output. SPD (Bushnaq, Braun and Sharkey, "Stochastic Parameter Decomposition", arXiv 2506.20790) and VPD (Bushnaq et al., "Interpreting Language Model Parameters", Goodfire, 2026: adversarial parameter decomposition) are both here, after their reference code (github.com/goodfire-ai/ param-decomp: tag v1 for SPD, nano_param_decomp for VPD).
More...
|
| class | pulsatrix::ComponentLinear |
| | A LinearModule split into C rank-one subcomponents: W ≈ V U, V (in, C) reading the input and U (C, out) writing the output, so subcomponent c is V[:, c] U[c, :] and its inner activation is h_c = (x V)_c. The bias is the target's, frozen. More...
|
| |
| struct | pulsatrix::DecompositionOptions |
| |
| struct | pulsatrix::DecompositionLoss |
| |
| class | pulsatrix::ParameterDecomposition |
| | Trains a decomposition of the ComponentLinear layers inside model against the model itself: the target is the model with every layer in target mode. More...
|
| |
| struct | pulsatrix::ComponentAlignment |
| | How well a layer's subcomponents line up with its target weight's rows, the input features (SPD's toy-model measure): for each input feature j, the best cosine over c between subcomponent c's row j, V[j, c] U[c, :], and the target's row j (MMCS), and that subcomponent's norm over the target row's (ML2R). More...
|
| |
| struct | pulsatrix::ImportanceStats |
| | Causal importance statistics of one layer over inputs. More...
|
| |
Parameter decomposition (FEAT-9): explaining a network in weight space. Each weight matrix is split into rank-one subcomponents, and training asks that, for every input, few of them be needed to compute the output. SPD (Bushnaq, Braun and Sharkey, "Stochastic Parameter Decomposition", arXiv 2506.20790) and VPD (Bushnaq et al., "Interpreting Language Model Parameters", Goodfire, 2026: adversarial parameter decomposition) are both here, after their reference code (github.com/goodfire-ai/ param-decomp: tag v1 for SPD, nano_param_decomp for VPD).