pulsatrix 1.1.0
Loading...
Searching...
No Matches
attnlrp_parity.hpp
Go to the documentation of this file.
1
6#pragma once
7
8#include <cstdint>
9#include <map>
10#include <string>
11#include <vector>
12
16
17namespace pulsatrix {
18
21 int64_t num_tokens = 0;
23 int64_t target = 0;
25 float token_correlation = 0.0f;
28 float key_correlation = 0.0f;
29 float value_correlation = 0.0f;
32 float max_relative_diff = 0.0f;
34 float relevance_sum = 0.0f;
35 float reference_sum = 0.0f;
36};
37
40 std::vector<AttnLrpSequenceResult> sequences;
43 float min_kv_correlation = 1.0f;
45 float max_relative_diff = 0.0f;
46 float threshold = 0.0f;
48 bool passed = false;
50 std::map<std::string, std::string> metadata;
51};
52
54[[nodiscard]] double PearsonCorrelation(const std::vector<float>& x, const std::vector<float>& y);
55
61
70[[nodiscard]] AttnLrpReport CompareToAttnLrp(CausalLM& model, const SafetensorsFile& reference,
71 const LRPRuleConfig& config = LxtAttnLrpConfig(),
72 float threshold = 0.99f);
73
74} // namespace pulsatrix
A Llama-family decoder-only language model built from a Hugging Face config, and loading its Hugging ...
Token embedding, num_hidden_layers pre-norm transformer blocks, a final RMSNorm and a language-model ...
Definition causal_lm.hpp:82
A parsed, fully validated safetensors file, held in memory or memory-mapped. Copies share the same by...
Definition safetensors.hpp:47
Selects which LRP rule variant a Module::propagate_relevance() call uses.
Definition acquisition_functions.hpp:16
AttnLrpReport CompareToAttnLrp(CausalLM &model, const SafetensorsFile &reference, const LRPRuleConfig &config=LxtAttnLrpConfig(), float threshold=0.99f)
Explains each ids.<k> sequence of an AttnLRP reference file (as written by tools/golden/make_attnlrp_...
double PearsonCorrelation(const std::vector< float > &x, const std::vector< float > &y)
Pearson correlation of two equal-length vectors; 0 if either is constant.
LRPRuleConfig LxtAttnLrpConfig()
The rule configuration that matches LXT's AttnLRP on a Hugging Face model: the epsilon rule with a ti...
Native safetensors reader and writer (roadmap IO-1): the one file format pulsatrix reads and writes f...
A model's comparison against an AttnLRP reference file.
Definition attnlrp_parity.hpp:39
float max_relative_diff
The largest max_relative_diff over all sequences (reported; not part of passed).
Definition attnlrp_parity.hpp:45
float threshold
Definition attnlrp_parity.hpp:46
float min_token_correlation
The lowest token, key and value correlations over all sequences.
Definition attnlrp_parity.hpp:42
bool passed
Every correlation above the threshold.
Definition attnlrp_parity.hpp:48
float min_kv_correlation
Definition attnlrp_parity.hpp:43
std::vector< AttnLrpSequenceResult > sequences
Definition attnlrp_parity.hpp:40
std::map< std::string, std::string > metadata
The reference file's metadata: model, revision, versions.
Definition attnlrp_parity.hpp:50
One reference sequence's comparison.
Definition attnlrp_parity.hpp:20
float value_correlation
Definition attnlrp_parity.hpp:29
float max_relative_diff
max |pulsatrix - LXT| over the tokens, divided by the largest |LXT| token relevance: catches a scale ...
Definition attnlrp_parity.hpp:32
float key_correlation
The lowest, over layers, Pearson correlation of the per-(KV head, token) relevance at k_proj's output...
Definition attnlrp_parity.hpp:28
float token_correlation
Pearson correlation of the per-token input relevance with LXT's.
Definition attnlrp_parity.hpp:25
int64_t num_tokens
Definition attnlrp_parity.hpp:21
float relevance_sum
Total input relevance, pulsatrix's and LXT's (both seeded with the logit).
Definition attnlrp_parity.hpp:34
int64_t target
The explained token at the last position (the reference's argmax).
Definition attnlrp_parity.hpp:23
float reference_sum
Definition attnlrp_parity.hpp:35
Configuration for LRP relevance propagation: which rule a module applies and its hyperparameters....
Definition lrp_rule_config.hpp:72