pulsatrix 1.1.0
Loading...
Searching...
No Matches
explanation_agreement.hpp
Go to the documentation of this file.
1
29#pragma once
30
31#include <cstdint>
32#include <vector>
33
36#include "pulsatrix/json.hpp"
38
39namespace pulsatrix {
40
45 bool magnitudes = true;
47 double top_fraction = 0.1;
49 int64_t permutations = 999;
50 uint64_t seed = 0;
52};
53
56 double spearman = 0.0;
57 double top_k_overlap = 0.0;
58 double cosine = 0.0;
59};
60
63 std::vector<InputAgreement> per_input;
65 double spearman = 0.0;
66 double top_k_overlap = 0.0;
67 double cosine = 0.0;
70 double cka = 0.0;
74 double mismatched_spearman = 0.0;
78 double p_value = 0.0;
79 int64_t permutations = 0;
80 int64_t inputs = 0;
81 int64_t features = 0;
82 int64_t top_k = 0;
83};
84
91[[nodiscard]] AgreementReport CompareExplanations(const std::vector<Attribution>& teacher, const std::vector<Attribution>& student,
92 const AgreementOptions& options = {});
93
95[[nodiscard]] AgreementReport ExplanationAgreement(const std::vector<Tensor>& inputs, const ExplainFn& teacher,
96 const ExplainFn& student, const AgreementOptions& options = {});
97
99[[nodiscard]] JsonValue ToJson(const AgreementReport& report);
100
101} // namespace pulsatrix
First-class explanation result type – values, method, and metadata together.
One JSON value: null, a boolean, a number, a string, an array or an object.
Definition json.hpp:22
Explanation-quality metrics in the Quantus families (roadmap XAI-5): faithfulness (deletion and inser...
A small strict JSON value, parser and deterministic writer (RFC 8259).
Definition acquisition_functions.hpp:16
JsonValue ToJson(const AttackResult &result, const AttackOptions &options)
The result as a pulsatrix.attack.v1 document, without the adversarial inputs.
AgreementReport CompareExplanations(const std::vector< Attribution > &teacher, const std::vector< Attribution > &student, const AgreementOptions &options={})
Compares two models' attributions of the same inputs, in the same order.
AgreementReport ExplanationAgreement(const std::vector< Tensor > &inputs, const ExplainFn &teacher, const ExplainFn &student, const AgreementOptions &options={})
Explains every input with both explainers, then CompareExplanations().
CkaEstimator
Which estimator of CKA to use.
Definition representation_similarity.hpp:45
@ Debiased
Unbiased HSIC in place of HSIC: near 0 for unrelated representations at any number of inputs....
std::function< Attribution(const Tensor &input)> ExplainFn
Explains input using the model in its current state.
Definition explanation_metrics.hpp:27
How similar two layers' representations of the same inputs are: linear CKA, RSA and SVCCA.
CompareExplanations()'s settings.
Definition explanation_agreement.hpp:42
uint64_t seed
Definition explanation_agreement.hpp:50
CkaEstimator estimator
Definition explanation_agreement.hpp:51
int64_t permutations
Relabelings of the student's inputs for the permutation test; 0 skips it.
Definition explanation_agreement.hpp:49
double top_fraction
The top-k overlap's k, as a fraction of the features: max(1, round(fraction × n)).
Definition explanation_agreement.hpp:47
bool magnitudes
Compare |attribution|, as XAI-3's checks do (a feature that counts against a class is still evidence)...
Definition explanation_agreement.hpp:45
CompareExplanations()'s result.
Definition explanation_agreement.hpp:62
double p_value
One-sided: the share of relabelings of the student's inputs, counting the real labeling,...
Definition explanation_agreement.hpp:78
int64_t features
Definition explanation_agreement.hpp:81
double spearman
The per-input values' means.
Definition explanation_agreement.hpp:65
double cka
Linear CKA of the (inputs × features) matrices; NaN with too few inputs for the estimator (4 debiased...
Definition explanation_agreement.hpp:70
double mismatched_spearman
The mean Spearman correlation between the student's map of one input and the teacher's map of another...
Definition explanation_agreement.hpp:74
double top_k_overlap
Definition explanation_agreement.hpp:66
std::vector< InputAgreement > per_input
Definition explanation_agreement.hpp:63
double cosine
Definition explanation_agreement.hpp:67
int64_t top_k
Definition explanation_agreement.hpp:82
int64_t permutations
Definition explanation_agreement.hpp:79
int64_t inputs
Definition explanation_agreement.hpp:80
How one input's two attributions compare.
Definition explanation_agreement.hpp:55
double spearman
Definition explanation_agreement.hpp:56
double cosine
Definition explanation_agreement.hpp:58
double top_k_overlap
Definition explanation_agreement.hpp:57