pulsatrix 1.1.0
Loading...
Searching...
No Matches
pulsatrix::ParameterDecomposition Class Reference

Trains a decomposition of the ComponentLinear layers inside model against the model itself: the target is the model with every layer in target mode. More...

#include <parameter_decomposition.hpp>

Public Types

using NoiseFn = std::function< std::vector< float >(const char *pass, int64_t layer, int64_t count)>
 Replaces the uniform draws, for tests: called with a pass name ("stochastic", "layerwise", "delta", "route"), the layer (or -1), and how many values are needed; returns them in [0, 1). For "route" one value per row, read as k = 1 + floor(v L), then L more per row ranking the layers (the k smallest are routed).
 

Public Member Functions

 ParameterDecomposition (Module &model, std::vector< ComponentLinear * > layers, DeviceBackend *backend, DecompositionOptions options={})
 
DecompositionLoss loss_and_backward (const Tensor &x)
 Every loss and its gradients, added to the layers' (zero them first).
 
Module & parameters_module ()
 The trainable parameters, for an optimizer's step() and zero_grad().
 
void set_p (float p)
 Importance minimality's exponent, for annealing (VPD: 2 down to 0.4).
 
const DecompositionOptions & options () const
 
const std::vector< ComponentLinear * > & layers () const
 
Tensor target_output (const Tensor &x)
 The target model's output for x.
 
std::vector< std::vector< float > > causal_importances (const Tensor &x)
 Each layer's causal importances (N, C) for x (lower clamp).
 
void set_noise (NoiseFn noise)
 
std::vector< std::vector< float > > & adversarial_sources ()
 VPD's persistent adversarial sources per layer, (N, C + 1), the last column the Δ mask's. Empty until the first step; set them to resume, or in tests.
 

Detailed Description

Trains a decomposition of the ComponentLinear layers inside model against the model itself: the target is the model with every layer in target mode.

  • The model can be any Module built from them (SequentialModule, ResidualModule, ...) whose backward() reaches every layer. Only the layers' parameters train (parameters_module()).
  • Causal importance comes from the target pass's inner activations. Masks use the lower clamp, g = clamp(z, 0, 1) with a leak below 0. SPD leaks in the forward pass too (0.01 z); VPD's is a straight-through leak only for gradients that raise g. The minimality loss uses the upper clamp, z in [0, 1] and 1 + leak (z - 1) above.
  • The gate input isn't detached (v1), so the importance losses reach V too.
  • One mask sample per step (S = 1), from the decomposition's own generator or set_noise().

Member Typedef Documentation

◆ NoiseFn

using pulsatrix::ParameterDecomposition::NoiseFn = std::function<std::vector<float>(const char* pass, int64_t layer, int64_t count)>

Replaces the uniform draws, for tests: called with a pass name ("stochastic", "layerwise", "delta", "route"), the layer (or -1), and how many values are needed; returns them in [0, 1). For "route" one value per row, read as k = 1 + floor(v L), then L more per row ranking the layers (the k smallest are routed).

Constructor & Destructor Documentation

◆ ParameterDecomposition()

pulsatrix::ParameterDecomposition::ParameterDecomposition ( Module &  model,
std::vector< ComponentLinear * >  layers,
DeviceBackend *  backend,
DecompositionOptions  options = {} 
)
Exceptions
std::invalid_argumentif layers is empty or holds a null.

Member Function Documentation

◆ adversarial_sources()

std::vector< std::vector< float > > & pulsatrix::ParameterDecomposition::adversarial_sources ( )
inline

VPD's persistent adversarial sources per layer, (N, C + 1), the last column the Δ mask's. Empty until the first step; set them to resume, or in tests.

◆ causal_importances()

std::vector< std::vector< float > > pulsatrix::ParameterDecomposition::causal_importances ( const Tensor &  x)

Each layer's causal importances (N, C) for x (lower clamp).

◆ layers()

const std::vector< ComponentLinear * > & pulsatrix::ParameterDecomposition::layers ( ) const
inline

◆ loss_and_backward()

DecompositionLoss pulsatrix::ParameterDecomposition::loss_and_backward ( const Tensor &  x)

Every loss and its gradients, added to the layers' (zero them first).

◆ options()

const DecompositionOptions & pulsatrix::ParameterDecomposition::options ( ) const
inline

◆ parameters_module()

Module & pulsatrix::ParameterDecomposition::parameters_module ( )

The trainable parameters, for an optimizer's step() and zero_grad().

◆ set_noise()

void pulsatrix::ParameterDecomposition::set_noise ( NoiseFn  noise)
inline

◆ set_p()

void pulsatrix::ParameterDecomposition::set_p ( float  p)
inline

Importance minimality's exponent, for annealing (VPD: 2 down to 0.4).

◆ target_output()

Tensor pulsatrix::ParameterDecomposition::target_output ( const Tensor &  x)

The target model's output for x.


The documentation for this class was generated from the following file: