pulsatrix 1.1.0
Loading...
Searching...
No Matches
attnlrp_parity.hpp File Reference

AttnLRP parity with LXT on a real language model (LLM-7): pulsatrix's relevance against LXT's own AttnLRP on the same Hugging Face model and tokens. More...

#include <cstdint>
#include <map>
#include <string>
#include <vector>
#include "pulsatrix/causal_lm.hpp"
#include "pulsatrix/lrp_rule_config.hpp"
#include "pulsatrix/safetensors.hpp"
Include dependency graph for attnlrp_parity.hpp:
This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Classes

struct  pulsatrix::AttnLrpSequenceResult
 One reference sequence's comparison. More...
 
struct  pulsatrix::AttnLrpReport
 A model's comparison against an AttnLRP reference file. More...
 

Namespaces

namespace  pulsatrix
 

Functions

double pulsatrix::PearsonCorrelation (const std::vector< float > &x, const std::vector< float > &y)
 Pearson correlation of two equal-length vectors; 0 if either is constant.
 
LRPRuleConfig pulsatrix::LxtAttnLrpConfig ()
 The rule configuration that matches LXT's AttnLRP on a Hugging Face model: the epsilon rule with a tiny epsilon, biases kept in the denominator as LXT's gradient * input keeps them in z (Qwen2's QKV biases), and LayerNorm with only its standard deviation held constant (LRPRuleConfig::layer_norm_detach_std; ESM-2, PLM-6).
 
AttnLrpReport pulsatrix::CompareToAttnLrp (CausalLM &model, const SafetensorsFile &reference, const LRPRuleConfig &config=LxtAttnLrpConfig(), float threshold=0.99f)
 Explains each ids.<k> sequence of an AttnLRP reference file (as written by tools/golden/make_attnlrp_reference.py) from the logit of target.<k> at the last position, seeded with that logit as LXT's examples do, and compares the input relevance and every layer's K/V-head relevance with the reference's.
 

Detailed Description

AttnLRP parity with LXT on a real language model (LLM-7): pulsatrix's relevance against LXT's own AttnLRP on the same Hugging Face model and tokens.