pulsatrix 1.1.0
Loading...
Searching...
No Matches
pulsatrix::LRPRuleConfig Struct Reference

Configuration for LRP relevance propagation: which rule a module applies and its hyperparameters. A module that does not implement the requested rule throws (see Module::supports_lrp_rule()) – the rule is never silently substituted. More...

#include <lrp_rule_config.hpp>

Public Attributes

float epsilon = 1e-6f
 Stabilizer added to every rule's denominator: z + epsilon * sign(z), sign(0) = +1.
 
LRPRule rule = LRPRule::Epsilon
 Which rule to apply.
 
float gamma = 0.25f
 Gamma rule: weight of the positive (or negative) part added to the weights.
 
float alpha = 1.0f
 AlphaBeta rule: weight of the positive contributions. Needs alpha, beta >= 0, alpha - beta == 1.
 
float beta = 0.0f
 AlphaBeta rule: weight of the negative contributions.
 
float low = 0.0f
 ZBox rule: lowest admissible input value (scalar, applied to every input element).
 
float high = 1.0f
 ZBox rule: highest admissible input value.
 
bool epsilon_bias_in_denominator = false
 Epsilon rule only, honoured by LinearModule / Conv2DModule: include the bias in the denominator, z = xW + b (Zennit's Epsilon). The default (false) keeps this codebase's conservative pre-bias z = xW, under which the bias absorbs no relevance.
 
bool layer_norm_detach_std = false
 LayerNormModule only: AttnLRP's normalization rule as LXT applies it. The standard deviation is held constant, and the rest, gamma (x - mean) / std + beta, is linear and takes the epsilon rule, its mean subtraction included (the bias in the denominator as epsilon_bias_in_denominator says). The default (false) passes relevance through unchanged, which is the same for RMSNorm but ignores LayerNorm's mean and bias. LxtAttnLrpConfig() sets it.
 
std::shared_ptr< const LRPRuleConfig > conv_rule
 The rule every Conv2DModule applies, wherever it sits, in place of this config (KS-9). Zennit maps rules by layer type through the whole network, and a residual block is one module here that passes a single config to the convolutions inside it, so a composite such as EpsilonPlus sets it ("Epsilon, and ZPlus for convolutions") instead of relying on each convolution being a top-level module. Empty: convolutions use this config.
 
LoRARelevance lora_relevance = LoRARelevance::Split
 LinearModule with an unmerged, enabled LoRA adapter only: split the relevance between the base and adapter paths (the default) or treat the layer as merged. A merged adapter can't be split: its update is part of W.
 

Detailed Description

Configuration for LRP relevance propagation: which rule a module applies and its hyperparameters. A module that does not implement the requested rule throws (see Module::supports_lrp_rule()) – the rule is never silently substituted.

Note
epsilon stays the first member so aggregate initialization LRPRuleConfig{0.01f} keeps meaning "epsilon rule, epsilon 0.01". It is also the denominator stabilizer of Gamma / AlphaBeta / ZBox (Zennit's stabilizer, default 1e-6 there as well).

Member Data Documentation

◆ alpha

float pulsatrix::LRPRuleConfig::alpha = 1.0f

AlphaBeta rule: weight of the positive contributions. Needs alpha, beta >= 0, alpha - beta == 1.

◆ beta

float pulsatrix::LRPRuleConfig::beta = 0.0f

AlphaBeta rule: weight of the negative contributions.

◆ conv_rule

std::shared_ptr<const LRPRuleConfig> pulsatrix::LRPRuleConfig::conv_rule

The rule every Conv2DModule applies, wherever it sits, in place of this config (KS-9). Zennit maps rules by layer type through the whole network, and a residual block is one module here that passes a single config to the convolutions inside it, so a composite such as EpsilonPlus sets it ("Epsilon, and ZPlus for convolutions") instead of relying on each convolution being a top-level module. Empty: convolutions use this config.

◆ epsilon

float pulsatrix::LRPRuleConfig::epsilon = 1e-6f

Stabilizer added to every rule's denominator: z + epsilon * sign(z), sign(0) = +1.

◆ epsilon_bias_in_denominator

bool pulsatrix::LRPRuleConfig::epsilon_bias_in_denominator = false

Epsilon rule only, honoured by LinearModule / Conv2DModule: include the bias in the denominator, z = xW + b (Zennit's Epsilon). The default (false) keeps this codebase's conservative pre-bias z = xW, under which the bias absorbs no relevance.

◆ gamma

float pulsatrix::LRPRuleConfig::gamma = 0.25f

Gamma rule: weight of the positive (or negative) part added to the weights.

◆ high

float pulsatrix::LRPRuleConfig::high = 1.0f

ZBox rule: highest admissible input value.

◆ layer_norm_detach_std

bool pulsatrix::LRPRuleConfig::layer_norm_detach_std = false

LayerNormModule only: AttnLRP's normalization rule as LXT applies it. The standard deviation is held constant, and the rest, gamma (x - mean) / std + beta, is linear and takes the epsilon rule, its mean subtraction included (the bias in the denominator as epsilon_bias_in_denominator says). The default (false) passes relevance through unchanged, which is the same for RMSNorm but ignores LayerNorm's mean and bias. LxtAttnLrpConfig() sets it.

◆ lora_relevance

LoRARelevance pulsatrix::LRPRuleConfig::lora_relevance = LoRARelevance::Split

LinearModule with an unmerged, enabled LoRA adapter only: split the relevance between the base and adapter paths (the default) or treat the layer as merged. A merged adapter can't be split: its update is part of W.

◆ low

float pulsatrix::LRPRuleConfig::low = 0.0f

ZBox rule: lowest admissible input value (scalar, applied to every input element).

◆ rule

LRPRule pulsatrix::LRPRuleConfig::rule = LRPRule::Epsilon

Which rule to apply.


The documentation for this struct was generated from the following file: