pulsatrix 1.1.0
Loading...
Searching...
No Matches
performance_views.hpp File Reference

Figures for the Performance question (KS-3): a reliability diagram of a classifier's calibration, and the error on each slice of the data with its interval and test. More...

#include <limits>
#include <string>
#include <string_view>
#include <vector>
#include "pulsatrix/performance.hpp"
#include "pulsatrix/viz/svg.hpp"
Include dependency graph for performance_views.hpp:

Go to the source code of this file.

Classes

struct  pulsatrix::ReliabilitySeries
 One classifier on a reliability diagram. More...
 
struct  pulsatrix::SliceErrorViewOptions
 RenderSliceErrorSvg()'s settings. More...
 

Namespaces

namespace  pulsatrix
 

Functions

ClassificationReport pulsatrix::ParseClassificationReport (std::string_view json)
 Reads a pulsatrix.classification_report.v1 document, as ToJson(ClassificationReport) writes it.
 
SliceErrorReport pulsatrix::ParseSliceErrorReport (std::string_view json)
 Reads a pulsatrix.slice_error.v1 document, as ToJson(SliceErrorReport) writes it.
 
std::string pulsatrix::RenderReliabilityDiagramSvg (const std::vector< ReliabilitySeries > &series, const SvgOptions &options={})
 A reliability diagram (Guo et al., ICML 2017): each confidence bin's accuracy against its mean confidence, one line per classifier. A calibrated classifier lies on the diagonal; points below it are overconfident, above it underconfident. The marks grow with the bin's share of the inputs, and a strip below shows that share, since a bin of three inputs can sit far from the diagonal by chance. The legend gives each one's ECE.
 
std::string pulsatrix::RenderReliabilityDiagramSvg (const ClassificationReport &report, const SvgOptions &options={})
 The diagram of one classifier.
 
std::string pulsatrix::RenderSliceErrorSvg (const SliceErrorReport &report, const SliceErrorViewOptions &view={}, const SvgOptions &options={})
 The error on each slice, worst first: a point with its interval, the error of the whole dataset as a dashed line, and each slice's p-value. Slices worse than the rest, after Bonferroni's correction, are red.
 

Detailed Description

Figures for the Performance question (KS-3): a reliability diagram of a classifier's calibration, and the error on each slice of the data with its interval and test.

Both read the documents performance.hpp writes, pulsatrix.classification_report.v1 and pulsatrix.slice_error.v1, so pulsatrix_svg draws them too. The figures follow svg.hpp's rules: standalone files, deterministic bytes, no font library, and a tooltip (<title>) on every mark.

Conformal coverage by slice is a slice-error figure of the misses EvaluateCoverage() reports, with the target miss rate alpha as the reference line.