pulsatrix 1.1.0
Loading...
Searching...
No Matches
steering.hpp File Reference

Steering a model by adding a direction to its residual stream (FEAT-7), with difference-of-means as the default direction, a featurizer's decoder direction as an option, and a report on how reliably steering works. More...

#include <cstdint>
#include <functional>
#include <vector>
#include "pulsatrix/causal_lm.hpp"
#include "pulsatrix/featurizer.hpp"
Include dependency graph for steering.hpp:

Go to the source code of this file.

Classes

struct  pulsatrix::SteeringOptions
 
struct  pulsatrix::SteeringReport
 MeasureSteering()'s result. More...
 

Namespaces

namespace  pulsatrix
 

Functions

std::vector< float > pulsatrix::DifferenceOfMeans (const std::vector< float > &positive, const std::vector< float > &negative, int64_t d)
 The difference of means: the mean of positive minus the mean of negative, both (N, d) row-major. The simple baseline that usually wins.
 
std::vector< float > pulsatrix::FeaturizerDirection (Featurizer &featurizer, int64_t feature)
 Feature feature's decoder direction, as a steering direction (unit length for a featurizer with unit-norm decoders).
 
HiddenStateHook pulsatrix::SteeringHook (int64_t position, std::vector< float > direction, float coefficient, std::vector< bool > rows={})
 A hook that adds coefficient * direction to the hidden states at position (0 is the embeddings, i layer i's output), for the rows in rows (every row when empty). Install it with CausalLM::set_hidden_state_hook() or EncoderLM::set_hidden_state_hook().
 
SteeringReport pulsatrix::MeasureSteering (const std::function< double(int64_t i, const HiddenStateHook &hook)> &behavior, int64_t num_inputs, int64_t position, const std::vector< float > &direction, const SteeringOptions &options={})
 Measures steering along direction, and along random directions of the same norm.
 

Detailed Description

Steering a model by adding a direction to its residual stream (FEAT-7), with difference-of-means as the default direction, a featurizer's decoder direction as an option, and a report on how reliably steering works.

Note
Why the report: steering vectors often look effective on average while failing on many inputs, some moving the wrong way, and they misgeneralize out of distribution (Tan et al., "Analyzing the Generalization and Reliability of Steering Vectors", NeurIPS 2024). On AxBench (Wu et al., arXiv 2501.17148) SAE features steer worse than simple baselines such as difference-of-means. So every steering result here comes with its per-input spread, the share of inputs it moves the wrong way, and random directions of the same size.