pulsatrix 1.1.0
Loading...
Searching...
No Matches
featurizer_metrics.hpp File Reference

How good a featurizer is (FEAT-3), after SAEBench (Karvonen et al., arXiv 2503.09532): reconstruction (explained variance, cosine, norm ratio, L0, dead and dense latents), loss recovered when the reconstruction is spliced back into the model, and feature absorption (Chanin et al., arXiv 2409.14507) with a linear probe as its baseline. More...

#include <cstdint>
#include <functional>
#include <utility>
#include <vector>
#include "pulsatrix/causal_lm.hpp"
#include "pulsatrix/featurizer.hpp"
#include "pulsatrix/mlp_hook.hpp"
Include dependency graph for featurizer_metrics.hpp:

Go to the source code of this file.

Classes

struct  pulsatrix::ReconstructionMetrics
 EvaluateReconstruction()'s result. More...
 
struct  pulsatrix::LossRecovered
 MeasureLossRecovered()'s result: the model's loss three ways. More...
 
struct  pulsatrix::AbsorptionOptions
 
struct  pulsatrix::AbsorptionResult
 FeatureAbsorption()'s result, on the held-out inputs. More...
 

Namespaces

namespace  pulsatrix
 

Functions

ReconstructionMetrics pulsatrix::EvaluateReconstruction (Featurizer &featurizer, const Tensor &x, double dense_rate=0.1, int64_t batch=4096)
 Encodes and reconstructs x, (N, input_dim), in batches of batch rows.
 
ReconstructionMetrics pulsatrix::EvaluatePrediction (Featurizer &featurizer, const Tensor &x, const Tensor &target, double dense_rate=0.1, int64_t batch=4096)
 EvaluateReconstruction for a transcoder: predict(x) against target, (N, output_dim). Explained variance is relative to the targets' variance.
 
HiddenStateHook pulsatrix::SpliceHook (Featurizer &featurizer, int64_t position, std::vector< bool > rows={})
 Splices the featurizer in at one position: a hook that replaces the hidden states there, (N, L, input_dim), by their reconstructions, and leaves other positions alone.
 
HiddenStateHook pulsatrix::AblationHook (int64_t position, std::vector< bool > rows={})
 Zero-ablates one position: the same rows set to 0 instead of reconstructed.
 
MlpHook pulsatrix::MlpSpliceHook (Featurizer &transcoder, std::vector< bool > rows={})
 Splices a transcoder in for an MLP (install it with EncoderBlock::set_mlp_hook() or TransformerBlock::set_mlp_hook()): the MLP's output rows are replaced by predict() of its input rows.
 
MlpHook pulsatrix::MlpAblationHook (std::vector< bool > rows={})
 Zero-ablates an MLP: its output rows set to 0.
 
LossRecovered pulsatrix::MeasureLossRecovered (Featurizer &featurizer, int64_t position, const std::function< double(const HiddenStateHook &)> &loss, const std::vector< bool > &rows={})
 Loss recovered, SAEBench's main downstream measure.
 
LossRecovered pulsatrix::MeasureMlpLossRecovered (Featurizer &transcoder, const std::function< double(const MlpHook &)> &loss, const std::vector< bool > &rows={})
 Loss recovered for a transcoder spliced in for an MLP: loss installs the MlpHook it is given on the MLP's block, runs the model, removes it and returns the loss. Ablation zeroes the MLP's output.
 
AbsorptionResult pulsatrix::FeatureAbsorption (Featurizer &featurizer, const Tensor &x, const std::vector< int > &labels, const AbsorptionOptions &options={})
 Feature absorption (Chanin et al.; SAEBench's version, in outline). A concept is absorbed when the features that stand for it stay silent on an input that has it, because a more specific feature fires instead and carries its direction: a "starts with S" feature that skips "short", which a "short" feature covers.
 

Detailed Description

How good a featurizer is (FEAT-3), after SAEBench (Karvonen et al., arXiv 2503.09532): reconstruction (explained variance, cosine, norm ratio, L0, dead and dense latents), loss recovered when the reconstruction is spliced back into the model, and feature absorption (Chanin et al., arXiv 2409.14507) with a linear probe as its baseline.

Note
SAEBench's authors find that these proxies don't reliably predict how useful a featurizer is. Report them with a baseline: the same metrics for a featurizer trained on a randomly initialized model's activations (NullModelBaseline, null_model_baseline.hpp), and, for a concept, a linear probe on the raw activations, which FeatureAbsorption always fits.