pulsatrix 1.1.0
Loading...
Searching...
No Matches
pulsatrix::EncoderExplainer Class Reference

AttnLRP for an ESM-2 EncoderLM: explains a target (EncoderTarget) as relevance on the input residues, with LXT's rules by default (the identity rule on LayerNorm and GELU, the uniform rule at attention's matmuls, epsilon elsewhere), so it matches LXT's AttnLRP. More...

#include <protein_explanations.hpp>

Public Member Functions

 EncoderExplainer (EncoderLM &model, const TextTokenizer &tokenizer, DeviceBackend *backend, LRPRuleConfig config=LxtAttnLrpConfig())
 
ResidueRelevance explain (std::string_view sequence, const EncoderTarget &target)
 Explains target on sequence.
 
AttributionGraph relevance_graph (std::string_view sequence, const EncoderTarget &target, const RelevanceGraphOptions &options)
 The explanation as an attribution graph over residues (VIZ-4's format, for Neuronpedia's and circuit-tracer's viewers): a node per layer and token, links carrying relevance between adjacent layers, and the output. The output's label is EncoderTarget::describe(), with the predicted token's probability for a masked or mutation target (1 for a head). Only config among options' rules is ignored: the explainer's own rules apply.
 
float evaluate (const std::vector< int64_t > &tokens, const EncoderTarget &target)
 The explained value on tokens (<cls> + residues + <eos>, some perhaps masked): the model run without keeping activations. The residue a masked target names is masked first.
 
std::vector< int64_t > tokens (std::string_view sequence) const
 <cls> + sequence + <eos> as token ids.
 
int64_t mask_id () const
 
EncoderLM & model ()
 

Static Public Member Functions

static const ExplainerCapabilities & capabilities ()
 What it answers and needs: its ExplainerCatalog() entry.
 

Detailed Description

AttnLRP for an ESM-2 EncoderLM: explains a target (EncoderTarget) as relevance on the input residues, with LXT's rules by default (the identity rule on LayerNorm and GELU, the uniform rule at attention's matmuls, epsilon elsewhere), so it matches LXT's AttnLRP.

Relevance at the output is the explained value itself, as LXT seeds it (value.backward()): the residues' relevance, with <cls>'s and <eos>'s, adds up to roughly that value; LRP through biases and norms isn't exactly conservative.

Constructor & Destructor Documentation

◆ EncoderExplainer()

pulsatrix::EncoderExplainer::EncoderExplainer ( EncoderLM &  model,
const TextTokenizer &  tokenizer,
DeviceBackend *  backend,
LRPRuleConfig  config = LxtAttnLrpConfig() 
)
Parameters
backendWhere the model runs. The model, tokenizer and backend must outlive this.

Member Function Documentation

◆ capabilities()

static const ExplainerCapabilities & pulsatrix::EncoderExplainer::capabilities ( )
inlinestatic

What it answers and needs: its ExplainerCatalog() entry.

◆ evaluate()

float pulsatrix::EncoderExplainer::evaluate ( const std::vector< int64_t > &  tokens,
const EncoderTarget &  target 
)

The explained value on tokens (<cls> + residues + <eos>, some perhaps masked): the model run without keeping activations. The residue a masked target names is masked first.

◆ explain()

ResidueRelevance pulsatrix::EncoderExplainer::explain ( std::string_view  sequence,
const EncoderTarget &  target 
)

Explains target on sequence.

Exceptions
std::invalid_argumentfor an empty sequence, a character the tokenizer doesn't know, a residue outside the sequence, a mutation whose wild type isn't the sequence's, or a head of the wrong size.

◆ mask_id()

int64_t pulsatrix::EncoderExplainer::mask_id ( ) const
inline

◆ model()

EncoderLM & pulsatrix::EncoderExplainer::model ( )
inline

◆ relevance_graph()

AttributionGraph pulsatrix::EncoderExplainer::relevance_graph ( std::string_view  sequence,
const EncoderTarget &  target,
const RelevanceGraphOptions &  options 
)

The explanation as an attribution graph over residues (VIZ-4's format, for Neuronpedia's and circuit-tracer's viewers): a node per layer and token, links carrying relevance between adjacent layers, and the output. The output's label is EncoderTarget::describe(), with the predicted token's probability for a masked or mutation target (1 for a head). Only config among options' rules is ignored: the explainer's own rules apply.

Note
Cost: explain(), then one layer LRP pass per layer and token.
Exceptions
std::invalid_argumentas explain(), or for a sequence longer than max_tokens - 2.

◆ tokens()

std::vector< int64_t > pulsatrix::EncoderExplainer::tokens ( std::string_view  sequence) const

<cls> + sequence + <eos> as token ids.


The documentation for this class was generated from the following file: