|
pulsatrix 1.1.0
|
Figures for the Performance question (KS-3): a reliability diagram of a classifier's calibration, and the error on each slice of the data with its interval and test. More...
#include <limits>#include <string>#include <string_view>#include <vector>#include "pulsatrix/performance.hpp"#include "pulsatrix/viz/svg.hpp"
Go to the source code of this file.
Classes | |
| struct | pulsatrix::ReliabilitySeries |
| One classifier on a reliability diagram. More... | |
| struct | pulsatrix::SliceErrorViewOptions |
| RenderSliceErrorSvg()'s settings. More... | |
Namespaces | |
| namespace | pulsatrix |
Functions | |
| ClassificationReport | pulsatrix::ParseClassificationReport (std::string_view json) |
Reads a pulsatrix.classification_report.v1 document, as ToJson(ClassificationReport) writes it. | |
| SliceErrorReport | pulsatrix::ParseSliceErrorReport (std::string_view json) |
Reads a pulsatrix.slice_error.v1 document, as ToJson(SliceErrorReport) writes it. | |
| std::string | pulsatrix::RenderReliabilityDiagramSvg (const std::vector< ReliabilitySeries > &series, const SvgOptions &options={}) |
| A reliability diagram (Guo et al., ICML 2017): each confidence bin's accuracy against its mean confidence, one line per classifier. A calibrated classifier lies on the diagonal; points below it are overconfident, above it underconfident. The marks grow with the bin's share of the inputs, and a strip below shows that share, since a bin of three inputs can sit far from the diagonal by chance. The legend gives each one's ECE. | |
| std::string | pulsatrix::RenderReliabilityDiagramSvg (const ClassificationReport &report, const SvgOptions &options={}) |
| The diagram of one classifier. | |
| std::string | pulsatrix::RenderSliceErrorSvg (const SliceErrorReport &report, const SliceErrorViewOptions &view={}, const SvgOptions &options={}) |
| The error on each slice, worst first: a point with its interval, the error of the whole dataset as a dashed line, and each slice's p-value. Slices worse than the rest, after Bonferroni's correction, are red. | |
Figures for the Performance question (KS-3): a reliability diagram of a classifier's calibration, and the error on each slice of the data with its interval and test.
Both read the documents performance.hpp writes, pulsatrix.classification_report.v1 and pulsatrix.slice_error.v1, so pulsatrix_svg draws them too. The figures follow svg.hpp's rules: standalone files, deterministic bytes, no font library, and a tooltip (<title>) on every mark.
Conformal coverage by slice is a slice-error figure of the misses EvaluateCoverage() reports, with the target miss rate alpha as the reference line.