JSON Documents¶
Every view in pulsatrix reads from a versioned JSON document, so an explanation can be saved,
redrawn later, or drawn by a program outside pulsatrix. The documents are part of
pulsatrix_core and need no build flag.
The ImGui widgets, the SVG renderer and any tool outside pulsatrix all read the same data, and a file format outlasts any one viewer.
#include <fstream>
#include <sstream>
#include "pulsatrix/viz/attribution_bar_chart.hpp" // the widget needs PULSATRIX_ENABLE_VIZ
#include "pulsatrix/viz/document.hpp"
// Save an explanation...
AttributionDocument doc = ToAttributionDocument(attribution);
std::ofstream("explanation.json") << ToJson(doc);
// ...and draw it later, possibly in another program.
std::stringstream text;
text << std::ifstream("explanation.json").rdbuf();
if (VizDocumentKind(text.str()) == "attribution") {
AttributionBarChart::Draw("Top features", ParseAttributionDocument(text.str()));
}
There are twenty kinds. Each has a struct, a writer (ToJson) and a reader (Parse<Kind>Document):
| Schema | Struct | Holds | Converts from and to |
|---|---|---|---|
pulsatrix.attribution.v1 |
AttributionDocument |
method, shape, row-major values, metadata |
Attribution |
pulsatrix.heatmap.v1 |
HeatmapDocument |
title, rows, cols, values, optional row_labels and col_labels |
HeatmapGrid |
pulsatrix.token_relevance.v1 |
TokenRelevanceDocument |
method, tokens, one relevance per token, target |
— |
pulsatrix.circuit_graph.v1 |
CircuitGraphDocument |
nodes (id, op_type, label, ablation_effect) and edges (from, to, weight) |
CircuitGraph |
pulsatrix.training_log.v1 |
TrainingLogDocument |
scalars (tag, steps, values) and the latest histograms per tag |
ImPlotMetricsSink; ReplayTrainingLog logs a saved run to any MetricsSink |
pulsatrix.feature_dashboard.v1 |
FeatureDashboardDocument |
One feature's source, feature_index, activation_density, max_activation, activation histogram and top_examples |
— |
pulsatrix.morris.v1 |
MorrisDocument |
target, num_trajectories, and per feature name, mu, mu_star, sigma, mu_star_conf |
MorrisResult |
pulsatrix.sobol.v1 |
SobolDocument |
target, num_samples, and per feature name, first_order, total_order and their _conf half-widths |
SobolResult |
pulsatrix.counterfactual.v1 |
CounterfactualDocument |
target, valid, output_before, output_after, and per feature its name, original and counterfactual value and distance scale |
CounterfactualResult |
pulsatrix.sensitivity.v1 |
SensitivityDocument |
target, the unchanged output, and per feature its name, value, low, high, output_low and output_high |
LocalSensitivityResult |
pulsatrix.partial_dependence.v1 |
PartialDependenceDocument |
method ("partial_dependence" or "ale"), feature, target, grid, partial_dependence, and optionally num_instances ICE curves (ice) and the instances' feature_values |
IceResult, AleResult |
pulsatrix.mutation_map.v1 |
MutationMapDocument |
title, sequence, first_position, method, alphabet, and row-major values (residues × alphabet) |
VariantScorer::single_mutant_scan |
pulsatrix.sequence_logo.v1 |
SequenceLogoDocument |
title, sequence, first_position, method, alphabet, and row-major probabilities (positions × alphabet, each row a distribution) |
ResidueLogProbs |
pulsatrix.contact_map.v1 |
ContactMapDocument |
title, sequence, first_position, method, predicted (L × L) and truth (L × L of 0, 1 and NaN, or empty) |
ContactMap |
pulsatrix.projection.v1 |
ProjectionDocument |
title, method, axes, explained_variance_ratio (one per axis, or empty), row-major coordinates (points × axes), and optional per-point labels, groups and values |
MakeProjectionDocument(pca, scores), or MakeProjectionDocument(coordinates, "umap") for UMAP or t-SNE output |
pulsatrix.persistence_diagram.v1 |
PersistenceDiagramDocument (viz/topology_documents.hpp) |
title, filtration, source, num_points, threshold, and per dimension its births, deaths (+∞ through nonfinite), certified error_bound, and optional p_values with their significance_threshold and bootstrap_band |
RipsResult or PersistenceDiagrams; AttachSignificance(UniversalNullResult), AttachBootstrapBand(BootstrapBandResult) |
pulsatrix.classification_report.v1 |
ClassificationReport (performance.hpp; read by ParseClassificationReport in viz/performance_views.hpp) |
n, classes, accuracy, log_loss, brier, ece, mce, macro_f1, reliability (per bin lower, upper, count, confidence, accuracy), confusion (classes × classes) and per_class (precision, recall, f1, support) |
EvaluateClassifier |
pulsatrix.slice_error.v1 |
SliceErrorReport (performance.hpp; read by ParseSliceErrorReport) |
n, error, test, alpha, and per slice its name, count, error, lower, upper, rest_error, p_value and worse |
PerSliceError, also of CoverageReport::misses |
pulsatrix.residue_tracks.v1 |
ResidueTracksDocument |
title, sequence, first_position, tracks (name, one value per residue, signed), features (name, start, end, category) and residue_ids |
— |
pulsatrix.series.v1 |
SeriesDocument (viz/series.hpp) |
title, x_label, y_label, x_scale and y_scale ("linear" or "log"), interval (what the intervals are), and panels, each a title and series: a name, x, y and optional lo and hi, one per point |
— (sweeps, trade-off curves, results over seeds) |
Token relevance for text¶
viz/text_relevance.hpp builds a token_relevance.v1 document from a real explanation, so its
pieces read as the original text:
MakeTokenRelevanceDocument(text, encoding, scores, method, target)shows each token as the text its offset covers, so byte-level BPE tokens appear decoded. Tokens that split one character (an emoji's bytes) become one piece with their scores summed. Text no token covers becomes unscored context. BOS and other inserted tokens appear as their token strings, or go tounassignedwhen left out.MakeWordRelevanceDocument(text, word_scores, method, target)takes the output ofAggregateToWords(per-word scores): each word is scored, and the spaces between words are unscored context.
The document grows three optional fields, all backward compatible, each written only when it
says something:
- granularity ("token" or "word")
- scored (one flag per piece; unscored pieces are context)
- unassigned (relevance that belongs to no piece shown)
The SVG token strip draws context as plain text, and so does the TokenRelevanceView ImGui
widget (viz/token_relevance_view.hpp).
pulsatrix_explain_text does all of this for a Hugging Face model in one step. It tokenizes,
runs AttnLRP from the most likely next token, and writes the figure or document:
pulsatrix_explain_text models/SmolLM2-135M "The Eiffel Tower is located in the city of" -o paris.svg
pulsatrix_explain_text models/SmolLM2-135M "The Eiffel Tower is located in the city of" --words -o paris.json
token_relevance_demo paris.json # the ImGui widget (PULSATRIX_ENABLE_VIZ)
For " Paris", the word view gives "Eiffel" +4.09, and every other word stays under 1.6 in magnitude ("The" is −1.60). The token view shows the relevance sits on "iffel"; "E" gets little.
The widgets read documents directly: AttributionBarChart, AttributionWaterfallChart,
SaliencyHeatmapView and CircuitGraphView each have a Draw overload for their document. For
the training dashboard, replay the log into an ImPlotMetricsSink once and draw that.
An attribution document looks like this:
{
"schema": "pulsatrix.attribution.v1",
"method": "integrated_gradients",
"shape": [2, 3],
"values": [0.5, -1.25, 0, 3, 0.1, -0.001],
"metadata": {
"baseline": "zero",
"steps": "50"
}
}
Examples of every kind are in
tests/fixtures/viz/.
Tests check that the writer reproduces each one byte for byte.
Format rules¶
- Schema and version. Every document is an object whose
"schema"ispulsatrix.<kind>.v<N>.VizDocumentKind()returns the kind, so a viewer can pick the reader. A reader accepts only its own major version. - Compatibility within a version. Version 1 only ever gains fields, and readers ignore fields they don't know, so an older reader can read a newer v1 file. Removing, renaming or changing the meaning of a field means v2.
-
NaN and infinity. JSON has no literal for them. A non-finite number is written as
null, and the top-level"nonfinite"object maps its JSON Pointer to"nan","inf"or"-inf":"values": [1, null], "nonfinite": { "/values/1": "nan" }The member appears only when the document has a non-finite number. A reader rejects a
nullnumber with no entry, and an entry that doesn't point at anullnumber, so no value is ever silently lost. - Numbers round-trip exactly. Each number is written with the shortest text that reads back as the samefloat(ordouble, for training-log values), formatted the way JavaScript prints numbers. Reading the file gives back the same bits. - Deterministic output. The same document always gives the same bytes on every platform: fixed member order, metadata sorted by key, two-space indent, and number arrays on one line. - Strict reading. The parser follows RFC 8259 strictly. It rejects comments, trailing commas, duplicate keys, invalid UTF-8 and nesting deeper than 256 levels. Every reader error names the JSON Pointer of the problem, for examplepulsatrix.heatmap.v1 document: /row_labels: needs one label per row, or none. - Text is UTF-8. Token strings must be valid UTF-8, so decode byte-level BPE tokens before writing them.
The parser is fuzzed (tools/fuzz/viz_document_fuzz.cpp). pulsatrix/json.hpp exposes the
underlying JsonValue, ParseJson and WriteJson for other uses.