Trains a decomposition of the ComponentLinear layers inside model against the model itself: the target is the model with every layer in target mode.
More...
#include <parameter_decomposition.hpp>
|
| using | NoiseFn = std::function< std::vector< float >(const char *pass, int64_t layer, int64_t count)> |
| | Replaces the uniform draws, for tests: called with a pass name ("stochastic", "layerwise", "delta", "route"), the layer (or -1), and how many values are needed; returns them in [0, 1). For "route" one value per row, read as k = 1 + floor(v L), then L more per row ranking the layers (the k smallest are routed).
|
| |
|
| | ParameterDecomposition (Module &model, std::vector< ComponentLinear * > layers, DeviceBackend *backend, DecompositionOptions options={}) |
| |
| DecompositionLoss | loss_and_backward (const Tensor &x) |
| | Every loss and its gradients, added to the layers' (zero them first).
|
| |
| Module & | parameters_module () |
| | The trainable parameters, for an optimizer's step() and zero_grad().
|
| |
| void | set_p (float p) |
| | Importance minimality's exponent, for annealing (VPD: 2 down to 0.4).
|
| |
| const DecompositionOptions & | options () const |
| |
| const std::vector< ComponentLinear * > & | layers () const |
| |
| Tensor | target_output (const Tensor &x) |
| | The target model's output for x.
|
| |
| std::vector< std::vector< float > > | causal_importances (const Tensor &x) |
| | Each layer's causal importances (N, C) for x (lower clamp).
|
| |
| void | set_noise (NoiseFn noise) |
| |
| std::vector< std::vector< float > > & | adversarial_sources () |
| | VPD's persistent adversarial sources per layer, (N, C + 1), the last column the Δ mask's. Empty until the first step; set them to resume, or in tests.
|
| |
Trains a decomposition of the ComponentLinear layers inside model against the model itself: the target is the model with every layer in target mode.
- The model can be any Module built from them (SequentialModule, ResidualModule, ...) whose backward() reaches every layer. Only the layers' parameters train (parameters_module()).
- Causal importance comes from the target pass's inner activations. Masks use the lower clamp,
g = clamp(z, 0, 1) with a leak below 0. SPD leaks in the forward pass too (0.01 z); VPD's is a straight-through leak only for gradients that raise g. The minimality loss uses the upper clamp, z in [0, 1] and 1 + leak (z - 1) above.
- The gate input isn't detached (v1), so the importance losses reach V too.
- One mask sample per step (S = 1), from the decomposition's own generator or set_noise().
◆ NoiseFn
Replaces the uniform draws, for tests: called with a pass name ("stochastic", "layerwise", "delta", "route"), the layer (or -1), and how many values are needed; returns them in [0, 1). For "route" one value per row, read as k = 1 + floor(v L), then L more per row ranking the layers (the k smallest are routed).
◆ ParameterDecomposition()
- Exceptions
-
| std::invalid_argument | if layers is empty or holds a null. |
◆ adversarial_sources()
| std::vector< std::vector< float > > & pulsatrix::ParameterDecomposition::adversarial_sources |
( |
| ) |
|
|
inline |
VPD's persistent adversarial sources per layer, (N, C + 1), the last column the Δ mask's. Empty until the first step; set them to resume, or in tests.
◆ causal_importances()
| std::vector< std::vector< float > > pulsatrix::ParameterDecomposition::causal_importances |
( |
const Tensor & |
x | ) |
|
Each layer's causal importances (N, C) for x (lower clamp).
◆ layers()
| const std::vector< ComponentLinear * > & pulsatrix::ParameterDecomposition::layers |
( |
| ) |
const |
|
inline |
◆ loss_and_backward()
Every loss and its gradients, added to the layers' (zero them first).
◆ options()
◆ parameters_module()
| Module & pulsatrix::ParameterDecomposition::parameters_module |
( |
| ) |
|
The trainable parameters, for an optimizer's step() and zero_grad().
◆ set_noise()
| void pulsatrix::ParameterDecomposition::set_noise |
( |
NoiseFn |
noise | ) |
|
|
inline |
◆ set_p()
| void pulsatrix::ParameterDecomposition::set_p |
( |
float |
p | ) |
|
|
inline |
Importance minimality's exponent, for annealing (VPD: 2 down to 0.4).
◆ target_output()
| Tensor pulsatrix::ParameterDecomposition::target_output |
( |
const Tensor & |
x | ) |
|
The target model's output for x.
The documentation for this class was generated from the following file: