pulsatrix 1.1.0
Loading...
Searching...
No Matches
lrp_rule_config.hpp
Go to the documentation of this file.
1
8#pragma once
9
10#include <memory>
11
12#include <string>
13
14namespace pulsatrix {
15
21enum class LRPRule {
24 Epsilon,
28 Gamma,
33 ZBox
34};
35
37enum class LoRARelevance {
44 Split,
46 Merged
47};
48
50inline std::string lrp_rule_name(LRPRule rule) {
51 switch (rule) {
53 return "epsilon";
54 case LRPRule::Gamma:
55 return "gamma";
57 return "alpha_beta";
58 case LRPRule::ZBox:
59 return "zbox";
60 }
61 return "unknown";
62}
63
74 float epsilon = 1e-6f;
78 float gamma = 0.25f;
80 float alpha = 1.0f;
82 float beta = 0.0f;
84 float low = 0.0f;
86 float high = 1.0f;
110 std::shared_ptr<const LRPRuleConfig> conv_rule;
115};
116
117} // namespace pulsatrix
Definition acquisition_functions.hpp:16
std::string lrp_rule_name(LRPRule rule)
Lower-case rule name ("epsilon", "gamma", "alpha_beta", "zbox") for messages/metadata.
Definition lrp_rule_config.hpp:50
LoRARelevance
How a LinearModule with an unmerged LoRA adapter (TRN-7, TRN-8) propagates relevance.
Definition lrp_rule_config.hpp:37
@ Merged
The rule runs on the adapted weight W + (alpha / r) A B, exactly as on a merged adapter.
LRPRule
The LRP rule family a module applies. Semantics follow Zennit 1.0.0 exactly (Anders et al....
Definition lrp_rule_config.hpp:21
@ AlphaBeta
Alpha-beta rule (Bach et al. 2015) with alpha - beta == 1; alpha 1, beta 0 is ZPlus.
Configuration for LRP relevance propagation: which rule a module applies and its hyperparameters....
Definition lrp_rule_config.hpp:72
float low
ZBox rule: lowest admissible input value (scalar, applied to every input element).
Definition lrp_rule_config.hpp:84
float epsilon
Stabilizer added to every rule's denominator: z + epsilon * sign(z), sign(0) = +1.
Definition lrp_rule_config.hpp:74
bool layer_norm_detach_std
LayerNormModule only: AttnLRP's normalization rule as LXT applies it. The standard deviation is held ...
Definition lrp_rule_config.hpp:101
std::shared_ptr< const LRPRuleConfig > conv_rule
The rule every Conv2DModule applies, wherever it sits, in place of this config (KS-9)....
Definition lrp_rule_config.hpp:110
float gamma
Gamma rule: weight of the positive (or negative) part added to the weights.
Definition lrp_rule_config.hpp:78
LoRARelevance lora_relevance
LinearModule with an unmerged, enabled LoRA adapter only: split the relevance between the base and ad...
Definition lrp_rule_config.hpp:114
float beta
AlphaBeta rule: weight of the negative contributions.
Definition lrp_rule_config.hpp:82
float high
ZBox rule: highest admissible input value.
Definition lrp_rule_config.hpp:86
float alpha
AlphaBeta rule: weight of the positive contributions. Needs alpha, beta >= 0, alpha - beta == 1.
Definition lrp_rule_config.hpp:80
bool epsilon_bias_in_denominator
Epsilon rule only, honoured by LinearModule / Conv2DModule: include the bias in the denominator,...
Definition lrp_rule_config.hpp:92
LRPRule rule
Which rule to apply.
Definition lrp_rule_config.hpp:76