pulsatrix 1.1.0
Loading...
Searching...
No Matches
probe_controls.hpp File Reference

Linear probes with controls (INT-3): a multi-class probe trainer, Hewitt and Liang's control task and selectivity, and a comparison against a baseline representation. More...

#include <cstdint>
#include <vector>
#include "pulsatrix/device_backend.hpp"
Include dependency graph for probe_controls.hpp:
This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Classes

struct  pulsatrix::ProbeOptions
 
struct  pulsatrix::ProbeResult
 A probe's quality on held-out rows. More...
 
struct  pulsatrix::ProbeSelectivity
 A probe on the real task beside the same probe on the control task. More...
 
struct  pulsatrix::ProbeGain
 A probe on a representation beside the same probe on a baseline representation. More...
 

Namespaces

namespace  pulsatrix
 

Functions

ProbeResult pulsatrix::TrainLinearProbe (const std::vector< float > &features, int64_t dim, const std::vector< int > &labels, int classes, const std::vector< bool > &train, const std::vector< bool > &test, DeviceBackend *backend, const ProbeOptions &options={})
 Trains a linear softmax probe (standardized features, AdamW, cross-entropy) on the rows marked in train and scores it on those marked in test.
 
std::vector< int > pulsatrix::ControlTaskLabels (const std::vector< int64_t > &token_types, const std::vector< int > &labels, int classes, uint64_t seed)
 Hewitt and Liang's control labels: each distinct value in token_types gets one label, drawn from the distribution of labels, and every row of that type carries it.
 
ProbeSelectivity pulsatrix::MeasureSelectivity (const std::vector< float > &features, int64_t dim, const std::vector< int > &labels, int classes, const std::vector< int64_t > &token_types, const std::vector< bool > &train, const std::vector< bool > &test, DeviceBackend *backend, const ProbeOptions &options={})
 Trains one probe on labels and one on ControlTaskLabels(token_types, labels, classes, options.seed), with the same features, split and options, and reports both.
 
ProbeGain pulsatrix::CompareToBaseline (const std::vector< float > &features, int64_t dim, const std::vector< float > &baseline, int64_t baseline_dim, const std::vector< int > &labels, int classes, const std::vector< bool > &train, const std::vector< bool > &test, DeviceBackend *backend, const ProbeOptions &options={})
 Trains the same probe on features and on baseline, rows matched, and reports how much better the representation does. A gain near zero means the probe reads nothing from the representation that the baseline (local sequence, a random model's layer, the raw input) doesn't already hold.
 

Detailed Description

Linear probes with controls (INT-3): a multi-class probe trainer, Hewitt and Liang's control task and selectivity, and a comparison against a baseline representation.

A probe's accuracy alone doesn't say the representation encodes a concept: the probe may have learned the task itself, for example by memorizing a label for each word. Hewitt and Liang ("Designing and Interpreting Probes with Control Tasks", EMNLP 2019) give each token type a random but fixed label and train the same probe on that. A control task can only be learned by memorizing types, so selectivity, the task accuracy minus the control accuracy, is what the representation adds beyond that.

When there are few types the control task is no control: a linear probe learns 20 amino acids' labels from any representation that encodes identity. Compare against a baseline representation instead, such as one-hot local sequence (SequenceWindowFeatures()) or the same layer of a randomly initialized model.