pulsatrix 1.1.0
Loading...
Searching...
No Matches
explanation_agreement.hpp File Reference

Whether two models explain the same inputs with the same evidence (roadmap KD-2): a distilled student against its teacher, a fine-tuned model against its base, or any two models that take the same input. More...

#include <cstdint>
#include <vector>
#include "pulsatrix/attribution.hpp"
#include "pulsatrix/explanation_metrics.hpp"
#include "pulsatrix/json.hpp"
#include "pulsatrix/representation_similarity.hpp"
Include dependency graph for explanation_agreement.hpp:

Go to the source code of this file.

Classes

struct  pulsatrix::AgreementOptions
 CompareExplanations()'s settings. More...
 
struct  pulsatrix::InputAgreement
 How one input's two attributions compare. More...
 
struct  pulsatrix::AgreementReport
 CompareExplanations()'s result. More...
 

Namespaces

namespace  pulsatrix
 

Functions

AgreementReport pulsatrix::CompareExplanations (const std::vector< Attribution > &teacher, const std::vector< Attribution > &student, const AgreementOptions &options={})
 Compares two models' attributions of the same inputs, in the same order.
 
AgreementReport pulsatrix::ExplanationAgreement (const std::vector< Tensor > &inputs, const ExplainFn &teacher, const ExplainFn &student, const AgreementOptions &options={})
 Explains every input with both explainers, then CompareExplanations().
 
JsonValue pulsatrix::ToJson (const AgreementReport &report)
 The report as a pulsatrix.explanation_agreement.v1 document. NaNs are written as null.
 

Detailed Description

Whether two models explain the same inputs with the same evidence (roadmap KD-2): a distilled student against its teacher, a fine-tuned model against its base, or any two models that take the same input.

A student can match its teacher's outputs while relying on different features. The two models' attributions of the same inputs are compared input by input:

  • Spearman rank correlation of the attributions, the comparison XAI-3's stability check and the randomization test use;
  • top-k overlap: the share of the teacher's k most relevant features that are also among the student's k most relevant (Quantus's TopKIntersection, with k a fraction of the features);
  • cosine similarity of the attributions;

and across inputs by the linear CKA of the two (inputs × features) attribution matrices (INT-2), which asks whether the maps vary from input to input in the same way.

Two maps can agree only because every map looks alike: an image model's relevance gathers in the middle of every image, a language model's on the last token. So the student's maps are also compared with the teacher's maps of other inputs, and a permutation test asks whether the matched pairs agree more than mismatched ones do. Only that excess is evidence of the same reasoning about each input.

tests/explanation_agreement_test.cpp checks every number against tools/generate_explanation_agreement_reference_values.py (scipy and numpy, the same shuffles).