69 [[nodiscard]] std::string
describe(std::string_view sequence)
const;
84 std::vector<std::vector<float>>
layers;
87 [[nodiscard]]
double total()
const;
135 [[nodiscard]] std::vector<int64_t>
tokens(std::string_view sequence)
const;
136 [[nodiscard]] int64_t
mask_id()
const {
return mask_id_; }
140 void Check(std::string_view sequence,
const EncoderTarget& target)
const;
143 std::vector<float>* logits);
145 std::vector<float>
Forward(
const std::vector<int64_t>&
tokens,
bool keep);
147 float Value(
const std::vector<float>& logits,
const std::vector<float>& hidden, int64_t T,
const EncoderTarget& target)
const;
153 int64_t mask_id_ = 0;
178 double top_fraction = 0.1);
205[[nodiscard]] std::string
AlignmentQuery(
const std::vector<FastaRecord>& alignment);
215 const std::function<
void(int64_t done, int64_t total)>& progress = {});
244 const std::vector<float>& relevance, int64_t steps = 20, int64_t random_orders = 5,
AttnLRP parity with LXT on a real language model (LLM-7): pulsatrix's relevance against LXT's own Att...
Vendor-agnostic compute/memory backend. CPUBackend, CUDABackend (Phase 1.5), and HIPBackend (Phase 1....
Definition device_backend.hpp:248
AttnLRP for an ESM-2 EncoderLM: explains a target (EncoderTarget) as relevance on the input residues,...
Definition protein_explanations.hpp:99
EncoderLM & model()
Definition protein_explanations.hpp:137
float evaluate(const std::vector< int64_t > &tokens, const EncoderTarget &target)
The explained value on tokens (<cls> + residues + <eos>, some perhaps masked): the model run without ...
static const ExplainerCapabilities & capabilities()
What it answers and needs: its ExplainerCatalog() entry.
Definition protein_explanations.hpp:102
int64_t mask_id() const
Definition protein_explanations.hpp:136
std::vector< int64_t > tokens(std::string_view sequence) const
<cls> + sequence + <eos> as token ids.
AttributionGraph relevance_graph(std::string_view sequence, const EncoderTarget &target, const RelevanceGraphOptions &options)
The explanation as an attribution graph over residues (VIZ-4's format, for Neuronpedia's and circuit-...
ResidueRelevance explain(std::string_view sequence, const EncoderTarget &target)
Explains target on sequence.
EncoderExplainer(EncoderLM &model, const TextTokenizer &tokenizer, DeviceBackend *backend, LRPRuleConfig config=LxtAttnLrpConfig())
A masked language model: token embeddings (with ESM's token-dropout scaling, when the config has it),...
Definition encoder_lm.hpp:109
A tokenizer: added tokens are split out of the raw text first (longest match, earliest first); each r...
Definition text_tokenizer.hpp:166
Masked-language-model encoders: ESM-2 protein language models (PLM-2), Hugging Face EsmForMaskedLM ch...
What each explainer answers: the nine question categories (Liao, Gruen and Miller,...
Definition acquisition_functions.hpp:16
RandomizationCheckResult RandomizationCheck(EncoderExplainer &explainer, std::string_view sequence, const EncoderTarget &target, uint64_t seed=0)
Cascading model randomization (Adebayo et al. 2018) over the encoder's own layers,...
ResidueDeletionCurve DeletionCheck(EncoderExplainer &explainer, std::string_view sequence, const EncoderTarget &target, const std::vector< float > &relevance, int64_t steps=20, int64_t random_orders=5, uint64_t seed=0)
Masks residues (<mask>) in order of how much they support the explained value (relevance of the value...
ResidueAgreement CompareResidueSignals(const std::vector< float > &relevance, const std::vector< float > &signal, double top_fraction=0.1)
Compares |relevance| with signal over the residues where both are finite; top_fraction picks k (at le...
const ExplainerCapabilities & ExplainerCapabilitiesOf(std::string_view name)
The catalog entry named name.
std::vector< float > DmsPositionSensitivity(const DmsVariants &variants, std::string_view sequence, int64_t offset=1)
Each residue's sensitivity in a deep mutational scan: minus the mean score of its single mutants (hig...
std::vector< float > AlignmentConservation(const std::vector< FastaRecord > &alignment)
Conservation of each query residue in a multiple sequence alignment whose first record is the query: ...
LRPRuleConfig LxtAttnLrpConfig()
The rule configuration that matches LXT's AttnLRP on a Hugging Face model: the epsilon rule with a ti...
std::vector< float > RelevanceProfile(EncoderExplainer &explainer, std::string_view sequence, const std::function< void(int64_t done, int64_t total)> &progress={})
How much the model draws on each residue when it predicts the others: the mean, over every residue's ...
@ Forward
KL(teacher ‖ student): the student covers every mode the teacher puts mass on.
std::string AlignmentQuery(const std::vector< FastaRecord > &alignment)
The query's residues in alignment's first record: its letters, uppercased.
Protein sequence input (PLM-2): ESM's tokenizer from its vocab.txt, and FASTA files.
Running ProteinGym's deep-mutational-scanning substitution benchmarks (PLM-3): read an assay,...
A language model's attribution graph from AttnLRP (VIZ-4): how relevance for one predicted token flow...
An attribution graph as Neuronpedia's schema (graph-schema.json) and circuit-tracer's create_graph_fi...
Definition attribution_graph.hpp:26
An assay's measured variants: mutant, DMS_score and DMS_score_bin columns.
Definition proteingym.hpp:30
What an explanation explains. Residues are numbered from 0 here.
Definition protein_explanations.hpp:44
static EncoderTarget ResidueHead(LinearHead head, int64_t residue)
static EncoderTarget ProteinHead(LinearHead head)
std::string describe(std::string_view sequence) const
For labels: "K11 masked: R", "K11R log-odds", "protein head", "head at K11".
Kind
Definition protein_explanations.hpp:45
@ Mutation
logit(token) - logit(wild_type) at residue, masked: a mutation's log-odds, the masked-marginal score ...
@ ResidueHead
A per-residue head on residue's representation, unmasked.
@ ProteinHead
A per-protein head on the mean of the residues' representations (<cls> and <eos> left out,...
@ MaskedToken
The logit of token at residue, with that residue masked.
int64_t residue
Definition protein_explanations.hpp:58
static EncoderTarget ForMutation(const Mutation &m, int64_t offset=1)
The mutation m; its position is numbered from offset, as variant tables do.
static EncoderTarget MaskedToken(int64_t residue, char token)
char wild_type
Definition protein_explanations.hpp:60
LinearHead head
Definition protein_explanations.hpp:61
Kind kind
Definition protein_explanations.hpp:57
char token
Definition protein_explanations.hpp:59
One explainer's declaration: what it answers and what it needs.
Definition explainer_capabilities.hpp:99
Configuration for LRP relevance propagation: which rule a module applies and its hyperparameters....
Definition lrp_rule_config.hpp:72
A linear head on the model's representations (last_hidden_state): weight . h + bias.
Definition protein_explanations.hpp:38
float bias
Definition protein_explanations.hpp:40
std::vector< float > weight
hidden_size values
Definition protein_explanations.hpp:39
One substitution, numbered from 1 as variant tables write it: A23G.
Definition variant_scoring.hpp:19
The randomization check's result: after randomizing layers[0..i], how similar the explanation stays (...
Definition protein_explanations.hpp:249
std::vector< std::string > layers
Definition protein_explanations.hpp:250
std::vector< float > similarity
Definition protein_explanations.hpp:251
Definition relevance_graph.hpp:20
How a per-residue explanation agrees with an independent per-residue signal.
Definition protein_explanations.hpp:159
int64_t compared
Residues where both are finite.
Definition protein_explanations.hpp:161
double top_overlap
The share of the top-k residues by |relevance| that are also top-k by the signal.
Definition protein_explanations.hpp:165
double spearman
Spearman correlation of |relevance| with the signal.
Definition protein_explanations.hpp:163
double chance_overlap
The share expected by chance, k / compared.
Definition protein_explanations.hpp:168
int64_t top
Definition protein_explanations.hpp:166
A deletion curve against its random baseline.
Definition protein_explanations.hpp:218
double random_auc
Definition protein_explanations.hpp:231
std::vector< float > scores
The explained value after masking that share, most relevant first.
Definition protein_explanations.hpp:222
double auc
Trapezoid areas under each curve times sign, so both measure how much of the value is left; a faithfu...
Definition protein_explanations.hpp:230
std::vector< float > fractions
Fractions of residues masked: 0, 1/steps, ..., 1.
Definition protein_explanations.hpp:220
std::vector< float > random_scores
The same, in random orders (their mean).
Definition protein_explanations.hpp:224
float sign
The sign of the unmasked value: +1, or -1 for a negative one such as most mutations' log-odds.
Definition protein_explanations.hpp:227
Relevance per residue for one explained value.
Definition protein_explanations.hpp:73
double total() const
The relevance of every token, <cls> and <eos> included.
float cls
Definition protein_explanations.hpp:80
std::string sequence
Definition protein_explanations.hpp:74
std::string target
Definition protein_explanations.hpp:75
std::vector< float > residues
One value per residue (the input tokens between <cls> and <eos>).
Definition protein_explanations.hpp:79
float value
The explained value: a logit, a log-odds or a head's output.
Definition protein_explanations.hpp:77
float eos
Definition protein_explanations.hpp:81
std::vector< std::vector< float > > layers
Relevance per token (<cls>, residues, <eos>) at the embeddings and after each layer: (layers + 1) row...
Definition protein_explanations.hpp:84
The tokenizer interface (TOK-1): a pipeline of normalizer, pre-tokenizer, model, post-processor and d...
Zero-shot variant effects from a protein language model (PLM-3): how much the model prefers a mutant ...