83 const std::vector<int64_t>& labels,
92 const Tensor& probabilities);
116 const std::vector<double>& targets,
double alpha = 0.1);
124 const std::vector<double>& upper,
125 const std::vector<double>& targets,
133 const std::vector<double>& predictions);
141 const std::vector<double>& lower,
142 const std::vector<double>& upper);
165 const std::vector<int64_t>& labels);
169 const std::vector<double>& targets);
One JSON value: null, a boolean, a number, a string, an array or an object.
Definition json.hpp:22
N-dimensional tensor. Owns its data buffer exclusively; a DeviceBackend* is injected (not owned) – th...
Definition tensor.hpp:29
A small strict JSON value, parser and deterministic writer (RFC 8259).
Definition acquisition_functions.hpp:16
std::vector< PredictionInterval > PredictionIntervals(const ConformalRegressor ®ressor, const std::vector< double > &predictions)
The intervals for a point regressor's predictions (absolute_residual).
ConformalScore
How a conformal classifier scores an example.
Definition conformal.hpp:41
@ Aps
The mass of the classes ranked at or above the true one: Adaptive Prediction Sets.
@ Lac
1 − p(true class): Least Ambiguous set-valued Classifier.
ConformalClassifier CalibrateConformalClassifier(const Tensor &probabilities, const std::vector< int64_t > &labels, const ConformalOptions &options={})
Calibrates on held-out examples the model wasn't trained on.
ConformalRegressor CalibrateConformalRegressor(const std::vector< double > &predictions, const std::vector< double > &targets, double alpha=0.1)
Calibrates intervals ŷ ± q̂ from a point regressor's predictions on held-out examples.
JsonValue ToJson(const AttackResult &result, const AttackOptions &options)
The result as a pulsatrix.attack.v1 document, without the adversarial inputs.
CoverageReport EvaluateCoverage(const std::vector< std::vector< int64_t > > &sets, const std::vector< int64_t > &labels)
std::vector< std::vector< int64_t > > PredictionSets(const ConformalClassifier &classifier, const Tensor &probabilities)
The prediction set of each input: the classes, most probable first.
ConformalRegressor CalibrateConformalQuantileRegressor(const std::vector< double > &lower, const std::vector< double > &upper, const std::vector< double > &targets, double alpha=0.1)
Calibrates CQR intervals [lo − q̂, hi + q̂] from a quantile regressor's lower and upper predictions,...
How often the sets or intervals held the truth, and how large they were.
Definition conformal.hpp:145
int64_t n
Definition conformal.hpp:146
int64_t empty
Classification only: how many sets were empty.
Definition conformal.hpp:152
std::vector< double > misses
Per input: 1 where the truth was missed, 0 where it was held; PerSliceError() on it tests each slice'...
Definition conformal.hpp:160
std::vector< int64_t > size_counts
Classification only: size_counts[s] sets had s classes, and coverage_by_size[s] of them held the trut...
Definition conformal.hpp:156
double coverage
The fraction that held the truth; compare with 1 − alpha.
Definition conformal.hpp:148
std::vector< double > coverage_by_size
Definition conformal.hpp:157
double mean_size
The mean set size, or the mean interval width.
Definition conformal.hpp:150
One prediction interval.
Definition conformal.hpp:105
double upper
Definition conformal.hpp:107
double lower
Definition conformal.hpp:106
N-dimensional tensor – owns a buffer via DeviceBackend*, RAII (Rule of Five).