pulsatrix 1.1.0
Loading...
Searching...
No Matches
explanation_robustness.hpp File Reference

How much an explanation changes when its input barely does (KS-4): the max- and average-sensitivity of Yeh et al. (NeurIPS 2019), the local Lipschitz estimate of Alvarez-Melis and Jaakkola (2018), and the sensitivity at the perturbation an attacker would try first, FGSM's. More...

#include <cstdint>
#include <optional>
#include <string>
#include "pulsatrix/adversarial.hpp"
#include "pulsatrix/explanation_metrics.hpp"
#include "pulsatrix/json.hpp"
Include dependency graph for explanation_robustness.hpp:

Go to the source code of this file.

Classes

struct  pulsatrix::SensitivityOptions
 ExplanationSensitivity()'s settings. More...
 
struct  pulsatrix::SensitivityReport
 ExplanationSensitivity()'s result. More...
 

Namespaces

namespace  pulsatrix
 

Functions

SensitivityReport pulsatrix::ExplanationSensitivity (const Tensor &input, const ExplainFn &explain, const PredictFn &predict, const DifferentiableClassifier *classifier, DeviceBackend *backend, const SensitivityOptions &options={})
 Measures how much explain's explanation of input changes under small perturbations.
 
JsonValue pulsatrix::ToJson (const SensitivityReport &report)
 The report as a pulsatrix.explanation_sensitivity.v1 document.
 

Detailed Description

How much an explanation changes when its input barely does (KS-4): the max- and average-sensitivity of Yeh et al. (NeurIPS 2019), the local Lipschitz estimate of Alvarez-Melis and Jaakkola (2018), and the sensitivity at the perturbation an attacker would try first, FGSM's.

An explanation that changes a lot under a change to the input too small to change the prediction says more about the explainer than the model: Ghorbani, Abid and Zou (AAAI 2019) changed saliency maps beyond recognition with imperceptible perturbations. These are the robustness metrics of Quantus (Hedström et al., JMLR 2023), computed the same way:

  • max- and average-sensitivity: ‖E(x') − E(x)‖₂ / ‖E(x)‖₂ over sampled x' near x, the largest and the mean;
  • local Lipschitz estimate: the largest ‖E(x') − E(x)‖₂ / ‖x' − x‖₂;
  • adversarial sensitivity: ‖E(x') − E(x)‖₂ / ‖E(x)‖₂ at x' = FGSM's step from x, the direction that changes the model's loss fastest.

The samples are uniform in the L∞ ball of radius radius times the input's standard deviation, and only those that keep the predicted class count: an explanation should change when the prediction does. tests/explanation_robustness_test.cpp checks the metrics against tools/generate_explanation_robustness_reference_values.py, which draws the same samples.