68 std::vector<std::pair<std::string, double>>
values;
71 [[nodiscard]] std::optional<double>
value(std::string_view key)
const;
110 std::vector<std::pair<std::string, std::string>>
settings;
112 std::vector<std::pair<std::string, double>>
metrics;
126 [[nodiscard]] std::optional<std::string>
setting(std::string_view key)
const;
149 std::vector<std::pair<std::string, std::string>>
settings;
248 std::vector<CheckResult> checks);
First-class explanation result type ā values, method, and metadata together.
One JSON value: null, a boolean, a number, a string, an array or an object.
Definition json.hpp:22
Base class for every layer type (LinearModule, Conv2DModule, activations, ...).
Definition module.hpp:78
N-dimensional tensor. Owns its data buffer exclusively; a DeviceBackend* is injected (not owned) ā th...
Definition tensor.hpp:29
Explanation-quality metrics in the Quantus families (roadmap XAI-5): faithfulness (deletion and inser...
Turns a question and a reason into a plan: which explainers can answer the question for this model,...
A small strict JSON value, parser and deterministic writer (RFC 8259).
Abstract base every layer subclasses ā NVI forward(), pure-virtual LRP contract.
Definition acquisition_functions.hpp:16
XaiReason
Adadi and Berrada's four reasons to explain a model (IEEE Access, 2018).
Definition explainer_capabilities.hpp:47
@ Justify
Justify a decision to someone it affects or who must sign it off.
std::function< Tensor(const Tensor &input)> PredictFn
The model as a function: input (with a leading batch dimension of 1) to outputs.
Definition explanation_metrics.hpp:24
CheckResult SliceErrorCheck(const SliceErrorReport &report)
The per_slice_error check's result for a report PerSliceError() made: Failed when a slice is worse th...
CheckOutcome
What a check found.
Definition explanation_report.hpp:48
@ NotRun
The check didn't run; CheckResult::summary says why.
@ Measured
The check ran and produced numbers with no pass or fail, such as an aggregate.
@ Passed
The check ran and the explanation met its criterion.
@ Failed
The check ran and the explanation didn't meet it.
std::string DescribeExplanation(const Explanation &explanation)
The explanation as plain text: the question, whether it supports the reason, the largest attributions...
CheckPriority
How much a check matters for this reason.
Definition explanation_planner.hpp:56
@ Recommended
Worth running; the reason doesn't depend on it.
XaiQuestion
The nine question categories of Liao, Gruen and Miller's XAI question bank (CHI 2020).
Definition explainer_capabilities.hpp:24
@ Why
Why this input gets this prediction.
JsonValue ToJson(const AttackResult &result, const AttackOptions &options)
The result as a pulsatrix.attack.v1 document, without the adversarial inputs.
Explanation BuildExplanation(const ExplanationRequest &request, const Tensor &input, const ExplainFn &explain, const PredictFn &predict, Module *model, const ExplanationCheckOptions &options={})
Explains input with explain and runs the checks the plan asks for.
std::vector< Caveat > DeriveCaveats(const Explanation &explanation)
The caveats an explanation carries, from the report alone:
CaveatSeverity
How much a caveat limits what the explanation supports.
Definition explanation_report.hpp:75
@ Warning
A recommended check failed, or a known failure mode applies to these settings.
@ Note
A limitation of the method worth knowing whenever it is read.
Explanation ReportExplanation(const ExplanationRequest &request, std::optional< Attribution > attribution, std::vector< CheckResult > checks)
The report for an explanation whose checks ran elsewhere: plans request.question and request....
std::function< Attribution(const Tensor &input)> ExplainFn
Explains input using the model in its current state.
Definition explanation_metrics.hpp:27
std::string_view to_string(XaiQuestion question)
One thing the explanation doesn't support, and why.
Definition explanation_report.hpp:86
CaveatSeverity severity
Definition explanation_report.hpp:89
std::string text
Definition explanation_report.hpp:92
std::string source
Where it came from: "check", "method", "input" or "plan".
Definition explanation_report.hpp:91
std::string id
A stable identifier: "check.faithfulness", "method.integrated_gradients_baseline".
Definition explanation_report.hpp:88
One check's result.
Definition explanation_report.hpp:60
std::optional< double > value(std::string_view key) const
The value named key, or std::nullopt.
std::string name
The PlannedCheck's name: "faithfulness", "randomization", ...
Definition explanation_report.hpp:62
std::vector< std::pair< std::string, double > > values
The numbers behind the outcome, in a fixed order: "p_value", "deletion_auc", ...
Definition explanation_report.hpp:68
CheckOutcome outcome
Definition explanation_report.hpp:64
CheckPriority priority
Definition explanation_report.hpp:63
std::string summary
One or two sentences: what was found, or why it didn't run.
Definition explanation_report.hpp:66
How BuildExplanation() runs the checks. The defaults are the published ones where there is one; each ...
Definition explanation_report.hpp:154
int64_t random_orderings
Faithfulness: random feature orderings to compare with. With 19, passing means beating every one of t...
Definition explanation_report.hpp:167
double alpha
Faithfulness: the largest p-value that passes.
Definition explanation_report.hpp:169
int64_t faithfulness_steps
Faithfulness: points on the deletion and insertion curves.
Definition explanation_report.hpp:161
int64_t stability_repeats
Stability: extra runs on the same input.
Definition explanation_report.hpp:172
double min_stability
Stability: the least Spearman correlation of attribution magnitudes that passes, both between repeate...
Definition explanation_report.hpp:179
double robustness_radius
Robustness: perturbations sampled uniformly within this fraction of the input's spread (Lā),...
Definition explanation_report.hpp:183
int64_t robustness_samples
Definition explanation_report.hpp:184
bool required_only
Run only the checks the reason requires, and mark the rest NotRun.
Definition explanation_report.hpp:156
Imputation imputation
Faithfulness: how removed features are filled. Use Imputation::NoisyLinear (ROAD) for images.
Definition explanation_report.hpp:164
uint64_t seed
Seed for the random orderings, the input noise and the re-initialized weights.
Definition explanation_report.hpp:158
double conservation_tolerance
Conservation: the largest gap between the attributions' sum and the output they explain,...
Definition explanation_report.hpp:191
double perturbation_scale
Stability: the noise's standard deviation, as a fraction of the input's.
Definition explanation_report.hpp:176
int64_t perturbations
Stability: runs on the input plus Gaussian noise, skipped for token and symbolic input.
Definition explanation_report.hpp:174
double max_sensitivity
Robustness: the largest relative change of the explanation that passes, both the largest over the sam...
Definition explanation_report.hpp:187
double max_random_similarity
Randomization and null model: the largest Spearman correlation between the trained and the randomized...
Definition explanation_report.hpp:195
What to explain, why, and what the caller has.
Definition explanation_report.hpp:130
XaiReason reason
Definition explanation_report.hpp:132
std::vector< std::string > slices
Definition explanation_report.hpp:147
std::vector< std::pair< std::string, std::string > > settings
Anything about how the explanation was made that the explainer's metadata leaves out.
Definition explanation_report.hpp:149
std::vector< int64_t > labels
For the per-slice error check: the true class of each dataset input, and the slice it belongs to.
Definition explanation_report.hpp:146
std::vector< Tensor > dataset
Inputs to aggregate explanations over, each shaped like the one explained.
Definition explanation_report.hpp:143
std::optional< Tensor > baseline
The baseline a complete explainer measured from (Integrated Gradients' path start,...
Definition explanation_report.hpp:141
int64_t target
The output (flat index into the prediction) the explanation is for.
Definition explanation_report.hpp:137
ModelTraits traits
Definition explanation_report.hpp:133
XaiQuestion question
Definition explanation_report.hpp:131
std::string method
The catalog name of the explainer the ExplainFn runs.
Definition explanation_report.hpp:135
An explanation and what it does and doesn't support (XAI-3).
Definition explanation_report.hpp:96
ModelTraits traits
Definition explanation_report.hpp:99
std::vector< std::pair< std::string, std::string > > settings
The explainer's metadata, sorted by key, then the caller's settings, then the checks' own settings un...
Definition explanation_report.hpp:110
std::optional< Attribution > aggregate
The mean absolute attribution over the dataset, when the aggregate check ran.
Definition explanation_report.hpp:117
std::string excluded_reason
Definition explanation_report.hpp:105
std::string method
The catalog name of the explainer: "IntegratedGradients".
Definition explanation_report.hpp:101
XaiReason reason
Definition explanation_report.hpp:98
std::optional< Attribution > attribution
What the explainer returned; empty for explainers that don't return an attribution.
Definition explanation_report.hpp:107
std::optional< std::string > setting(std::string_view key) const
The value of setting key, or std::nullopt.
std::vector< Caveat > caveats
DeriveCaveats(*this): blocking first, then warnings, then notes.
Definition explanation_report.hpp:119
const CheckResult * check(std::string_view name) const
The result of the check named name, or nullptr.
int plan_rank
Its place in the plan for this question, reason and model, from 1; 0 if the planner excluded it,...
Definition explanation_report.hpp:104
bool supports_reason() const
Whether nothing blocks using it for its reason.
std::vector< CheckResult > checks
One result per check the plan asks for, required first, plus any extra checks the caller ran.
Definition explanation_report.hpp:115
std::vector< std::pair< std::string, double > > metrics
Properties of the attribution itself: "sparseness" and "complexity".
Definition explanation_report.hpp:112
XaiQuestion question
Definition explanation_report.hpp:97
How a removed feature is filled in when perturbing an input.
Definition explanation_metrics.hpp:55
What the caller has: the model, its inputs and its data.
Definition explanation_planner.hpp:32
PerSliceError()'s result.
Definition performance.hpp:155