pulsatrix 1.1.0
Loading...
Searching...
No Matches
explanation_report.hpp File Reference

An explanation that says what it supports (XAI-3): the attribution together with the question and reason it was made for, the method and its settings, the results of the checks the plan asks for, and caveats derived from those results and from the method's known failure modes. More...

#include <cstdint>
#include <optional>
#include <string>
#include <string_view>
#include <utility>
#include <vector>
#include "pulsatrix/attribution.hpp"
#include "pulsatrix/explanation_metrics.hpp"
#include "pulsatrix/explanation_planner.hpp"
#include "pulsatrix/json.hpp"
#include "pulsatrix/module.hpp"
#include "pulsatrix/performance.hpp"
Include dependency graph for explanation_report.hpp:

Go to the source code of this file.

Classes

struct  pulsatrix::CheckResult
 One check's result. More...
 
struct  pulsatrix::Caveat
 One thing the explanation doesn't support, and why. More...
 
struct  pulsatrix::Explanation
 An explanation and what it does and doesn't support (XAI-3). More...
 
struct  pulsatrix::ExplanationRequest
 What to explain, why, and what the caller has. More...
 
struct  pulsatrix::ExplanationCheckOptions
 How BuildExplanation() runs the checks. The defaults are the published ones where there is one; each is recorded in Explanation::settings. More...
 

Namespaces

namespace  pulsatrix
 

Enumerations

enum class  pulsatrix::CheckOutcome { pulsatrix::Passed , pulsatrix::Failed , pulsatrix::Measured , pulsatrix::NotRun }
 What a check found. More...
 
enum class  pulsatrix::CaveatSeverity { pulsatrix::Blocking , pulsatrix::Warning , pulsatrix::Note }
 How much a caveat limits what the explanation supports. More...
 

Functions

Explanation pulsatrix::BuildExplanation (const ExplanationRequest &request, const Tensor &input, const ExplainFn &explain, const PredictFn &predict, Module *model, const ExplanationCheckOptions &options={})
 Explains input with explain and runs the checks the plan asks for.
 
Explanation pulsatrix::ReportExplanation (const ExplanationRequest &request, std::optional< Attribution > attribution, std::vector< CheckResult > checks)
 The report for an explanation whose checks ran elsewhere: plans request.question and request.reason, matches checks to the planned ones by name (each takes the planned priority), marks the planned checks missing from checks NotRun, and derives the caveats.
 
CheckResult pulsatrix::SliceErrorCheck (const SliceErrorReport &report)
 The per_slice_error check's result for a report PerSliceError() made: Failed when a slice is worse than the rest, Passed otherwise. For ReportExplanation(), or a model BuildExplanation() can't score, such as a regressor.
 
std::vector< Caveat > pulsatrix::DeriveCaveats (const Explanation &explanation)
 The caveats an explanation carries, from the report alone:
 
std::string_view pulsatrix::to_string (CheckOutcome outcome)
 
std::string_view pulsatrix::to_string (CaveatSeverity severity)
 
JsonValue pulsatrix::ToJson (const Explanation &explanation)
 The explanation as a pulsatrix.explanation.v1 document, attribution values included.
 
std::string pulsatrix::DescribeExplanation (const Explanation &explanation)
 The explanation as plain text: the question, whether it supports the reason, the largest attributions, the checks and the caveats.
 

Detailed Description

An explanation that says what it supports (XAI-3): the attribution together with the question and reason it was made for, the method and its settings, the results of the checks the plan asks for, and caveats derived from those results and from the method's known failure modes.

BuildExplanation() runs an attribution explainer and every check ExplanationPlanner (XAI-2) plans for it that pulsatrix can run on one input: faithfulness against random orderings, stability, conservation, the randomization test, the null-model baseline and aggregation over a dataset. ReportExplanation() builds the same report from checks run elsewhere, for explainers that don't return an attribution (a probe, persistent homology). Either way the caveats follow from the report alone (DeriveCaveats()), so a stored report can be re-checked.

ExplanationRequest request;
request.question = XaiQuestion::Why;
request.reason = XaiReason::Justify;
request.traits.access = ModelAccess::Gradients;
request.traits.can_reinitialize = true;
request.method = "IntegratedGradients";
request.baseline = zeros; // what IG integrated from, for the conservation check
const Explanation e = BuildExplanation(request, x,
[&](const Tensor& in) { return ig.explain(ctx, in, zeros, 0, 64, &backend); },
[&](const Tensor& in) { return model.forward(in); }, &model);
if (!e.supports_reason()) std::cout << DescribeExplanation(e); // the blocking caveats say why
std::string DescribeExplanation(const Explanation &explanation)
The explanation as plain text: the question, whether it supports the reason, the largest attributions...