|
pulsatrix 1.1.0
|
Timing, benchmark reports (pulsatrix.benchmark.v1) and regression comparison.
More...
#include <cstdint>#include <functional>#include <map>#include <string>#include <string_view>#include <vector>
Go to the source code of this file.
Classes | |
| struct | pulsatrix::TimingStats |
| Summary of repeated timings, in milliseconds. More... | |
| struct | pulsatrix::BenchmarkRecord |
| One measured quantity. More... | |
| struct | pulsatrix::BenchmarkReport |
A run of benchmarks: pulsatrix.benchmark.v1. More... | |
| struct | pulsatrix::CompareOptions |
| How CompareBenchmarks decides that a candidate got worse. More... | |
| struct | pulsatrix::BenchmarkComparison |
| One record compared across two builds. More... | |
Namespaces | |
| namespace | pulsatrix |
Functions | |
| TimingStats | pulsatrix::SummarizeTimings (std::vector< double > samples_ms) |
| Summarizes timings. Percentiles interpolate linearly between the sorted samples. | |
| TimingStats | pulsatrix::TimeIt (const std::function< void()> &fn, int warmup, int repeats) |
Runs fn warmup times untimed, then repeats times timed with a steady clock. | |
| std::string | pulsatrix::ToJson (const BenchmarkReport &report) |
| BenchmarkReport | pulsatrix::ParseBenchmarkReport (std::string_view json) |
| Reads a report, following the viz document rules (docs/visualization): schema and version checks, unknown fields ignored, "nonfinite" for NaN and infinity, and errors that name the JSON Pointer of the problem. | |
| std::vector< BenchmarkComparison > | pulsatrix::CompareBenchmarks (const std::vector< BenchmarkReport > &baseline_runs, const std::vector< BenchmarkReport > &candidate_runs, const CompareOptions &options={}) |
| Compares a candidate build with a baseline, record by record (matched on name and device). Several reports per side are the rounds of an interleaved A/B run; each side's value is the median over its rounds, so one noisy round doesn't decide, and with three or more rounds per side a time regresses only if its rounds also separate (CompareOptions::significance). | |
Timing, benchmark reports (pulsatrix.benchmark.v1) and regression comparison.
The pulsatrix_bench tool (tools/bench/) uses these to measure training step time, explanation time and LRP conservation error on every backend a build has, and to compare two builds. They are public so a project can benchmark its own models the same way.