|
| | ComponentLinear (LinearModule &target, int64_t num_components, DeviceBackend *backend, int64_t gate_hidden=16, uint64_t seed=0) |
| | Copies target's weight and bias, frozen.
|
| |
| int64_t | in_features () const |
| |
| int64_t | out_features () const |
| |
| int64_t | num_components () const |
| |
| int64_t | gate_hidden () const |
| |
| void | use_target () |
| | Later forward() calls use the target weight.
|
| |
| void | use_components (std::vector< float > masks, std::vector< float > delta_masks={}) |
| | Later forward() calls use the subcomponents under masks (N, C), and the Δ weight under delta_masks (N,) when given. Sizes are checked at forward().
|
| |
| bool | uses_target () const |
| |
| const std::vector< float > & | mask_grad () const |
| | dL/dm from the last backward() in components mode, (N, C).
|
| |
| const std::vector< float > & | delta_mask_grad () const |
| | dL/dm_Δ from the last backward() with a Δ mask, (N,).
|
| |
| std::vector< float > | gate_outputs () |
| | The gates' outputs z (N, C) for the input of the last target-mode forward(); the decomposition turns them into causal importances.
|
| |
| void | gate_backward (const std::vector< float > &grad_z) |
| | Backward from dL/dz (N, C) for the last gate_outputs(): into the gates' parameters and, through the inner activations, into V.
|
| |
| std::vector< float > | component_weight () |
| | The subcomponents' sum V U, (in, out) as LinearModule stores weights.
|
| |
| const std::vector< float > & | target_weight () const |
| | The frozen target weight, (in, out).
|
| |
| LinearModule & | V () |
| |
| LinearModule & | U () |
| |
| Tensor | backward (const Tensor &grad_output) override |
| | Computes the gradient w.r.t. this module's input, given the gradient w.r.t. its output. Must be called after forward() – uses state cached from that call.
|
| |
| Tensor | propagate_relevance (const Tensor &relevance_out, const LRPRuleConfig &config) override |
| | Through the weight in use: the target's, or the masked subcomponents (and Δ).
|
| |
| OpType | op_type () const override |
| | This module's operation-category tag, for ComputationGraph node tagging.
|
| |
| std::vector< NamedParamRef > | named_parameters () override |
| | This module's trainable parameters, each with its hierarchical name – the one place a module declares its parameters (roadmap FND-1).
|
| |
| std::optional< DeviceType > | compute_device () const override |
| | The device this module computes on, so forward() can reject an input on another device before any kernel sees it (roadmap FND-8).
|
| |
| void | release_activations () override |
| | Frees the tensors forward() kept for backward() and the relevance passes; containers release their children too. Inference that only needs the output calls it after each forward, so a deep model holds one layer's activations at a time (a 1024-token ESM-2 650M pass otherwise keeps over 10 GB of them).
|
| |
| virtual | ~Module ()=default |
| |
| Tensor | forward (const Tensor &input) |
| | Runs this module's forward computation.
|
| |
| std::pair< Tensor, NodeId > | forward_traced (const Tensor &input, NodeId input_node, ComputationGraph &graph, Autograd &autograd) |
| | Runs forward() while also registering a ComputationGraph node (tagged with this module's op_type(), parented to input_node) and wiring an Autograd backward function that reuses this module's own backward() – the opt-in traced/explainable path, per Phase 2 Mission 0.
|
| |
| virtual bool | supports_lrp_rule (LRPRule rule) const |
| | Whether propagate_relevance() implements rule (no silent fallback: callers such as ExplainerContext::relevance_pass() throw rather than run a module on a rule it does not implement).
|
| |
| virtual std::vector< NamedBufferRef > | named_buffers () |
| | This module's buffers: state a checkpoint must save that no optimizer updates.
|
| |
| virtual std::vector< NamedModuleRef > | named_children () |
| | This module's direct sub-modules, each under the prefix its parameters carry in named_parameters(), in the same order (TRN-7: how ApplyLora() finds a model's Linear layers). Default: none. NamedModules() walks the whole tree.
|
| |
| virtual std::vector< ParamRef > | parameters () |
| | This module's trainable parameters and their gradients, for an optimizer to update uniformly across module types.
|
| |
| void | set_requires_grad (bool requires_grad, const std::string &prefix="") |
| | Freezes (false) or unfreezes (true) parameters by name (roadmap FND-2).
|
| |
| virtual void | set_training (bool training) |
| | Sets this module's training/eval mode. Defaults to training (matches every mainstream framework's Module default).
|
| |
| bool | is_training () const |
| | Whether this module is currently in training mode.
|
| |
A LinearModule split into C rank-one subcomponents: W ≈ V U, V (in, C) reading the input and U (C, out) writing the output, so subcomponent c is V[:, c] U[c, :] and its inner activation is h_c = (x V)_c. The bias is the target's, frozen.
It runs in one of two modes, chosen before each forward():
- Target:
y = x W + b with the original weight, and the input is kept for causal importance (causal_importance()).
- Components:
y = ((x V) ⊙ m) U + b under a mask m (N, C). With a Δ mask (N,) (VPD), the rest of the weight, Δ = W - V U, is added back under it: y = ((x V) ⊙ (m - m_Δ)) U + m_Δ (x W) + b. Masks of 1 then give the target exactly.
backward() returns the input's gradient, adds to V's and U's, and keeps the masks' gradients (mask_grad(), delta_mask_grad()) for the decomposition to carry into the causal importances.
Causal importance (SPD): each subcomponent has its own small MLP from its inner activation to a scalar, z_c = w_out,c · GELU(w_in,c h_c + b_in,c) + b_out,c, hidden size gate_hidden. Parameters: V.weight, U.weight, gate.in_weight, gate.in_bias, gate.out_weight, gate.out_bias.