AttnLRP for an ESM-2 EncoderLM: explains a target (EncoderTarget) as relevance on the input residues, with LXT's rules by default (the identity rule on LayerNorm and GELU, the uniform rule at attention's matmuls, epsilon elsewhere), so it matches LXT's AttnLRP.
More...
#include <protein_explanations.hpp>
|
| | EncoderExplainer (EncoderLM &model, const TextTokenizer &tokenizer, DeviceBackend *backend, LRPRuleConfig config=LxtAttnLrpConfig()) |
| |
| ResidueRelevance | explain (std::string_view sequence, const EncoderTarget &target) |
| | Explains target on sequence.
|
| |
| AttributionGraph | relevance_graph (std::string_view sequence, const EncoderTarget &target, const RelevanceGraphOptions &options) |
| | The explanation as an attribution graph over residues (VIZ-4's format, for Neuronpedia's and circuit-tracer's viewers): a node per layer and token, links carrying relevance between adjacent layers, and the output. The output's label is EncoderTarget::describe(), with the predicted token's probability for a masked or mutation target (1 for a head). Only config among options' rules is ignored: the explainer's own rules apply.
|
| |
| float | evaluate (const std::vector< int64_t > &tokens, const EncoderTarget &target) |
| | The explained value on tokens (<cls> + residues + <eos>, some perhaps masked): the model run without keeping activations. The residue a masked target names is masked first.
|
| |
| std::vector< int64_t > | tokens (std::string_view sequence) const |
| | <cls> + sequence + <eos> as token ids.
|
| |
| int64_t | mask_id () const |
| |
| EncoderLM & | model () |
| |
AttnLRP for an ESM-2 EncoderLM: explains a target (EncoderTarget) as relevance on the input residues, with LXT's rules by default (the identity rule on LayerNorm and GELU, the uniform rule at attention's matmuls, epsilon elsewhere), so it matches LXT's AttnLRP.
Relevance at the output is the explained value itself, as LXT seeds it (value.backward()): the residues' relevance, with <cls>'s and <eos>'s, adds up to roughly that value; LRP through biases and norms isn't exactly conservative.
◆ EncoderExplainer()
- Parameters
-
| backend | Where the model runs. The model, tokenizer and backend must outlive this. |
◆ capabilities()
◆ evaluate()
| float pulsatrix::EncoderExplainer::evaluate |
( |
const std::vector< int64_t > & |
tokens, |
|
|
const EncoderTarget & |
target |
|
) |
| |
The explained value on tokens (<cls> + residues + <eos>, some perhaps masked): the model run without keeping activations. The residue a masked target names is masked first.
◆ explain()
Explains target on sequence.
- Exceptions
-
| std::invalid_argument | for an empty sequence, a character the tokenizer doesn't know, a residue outside the sequence, a mutation whose wild type isn't the sequence's, or a head of the wrong size. |
◆ mask_id()
| int64_t pulsatrix::EncoderExplainer::mask_id |
( |
| ) |
const |
|
inline |
◆ model()
| EncoderLM & pulsatrix::EncoderExplainer::model |
( |
| ) |
|
|
inline |
◆ relevance_graph()
The explanation as an attribution graph over residues (VIZ-4's format, for Neuronpedia's and circuit-tracer's viewers): a node per layer and token, links carrying relevance between adjacent layers, and the output. The output's label is EncoderTarget::describe(), with the predicted token's probability for a masked or mutation target (1 for a head). Only config among options' rules is ignored: the explainer's own rules apply.
- Note
- Cost: explain(), then one layer LRP pass per layer and token.
- Exceptions
-
| std::invalid_argument | as explain(), or for a sequence longer than max_tokens - 2. |
◆ tokens()
| std::vector< int64_t > pulsatrix::EncoderExplainer::tokens |
( |
std::string_view |
sequence | ) |
const |
<cls> + sequence + <eos> as token ids.
The documentation for this class was generated from the following file: