62[[nodiscard]] std::vector<std::pair<std::string, int64_t>>
ConceptCounts(
const std::vector<AnnotatedProtein>& proteins);
74 std::vector<std::vector<float>>
values;
76 [[nodiscard]]
const float*
row(
size_t layer_index, int64_t residue)
const {
90 const std::vector<std::string>& sequences,
const std::vector<int64_t>& layers,
91 int64_t max_tokens = 1024);
128 int64_t batch = 4096);
154 const std::vector<double>& thresholds = {0.0, 0.15, 0.5, 0.6, 0.8});
Vendor-agnostic compute/memory backend. CPUBackend, CUDABackend (Phase 1.5), and HIPBackend (Phase 1....
Definition device_backend.hpp:248
A masked language model: token embeddings (with ESM's token-dropout scaling, when the config has it),...
Definition encoder_lm.hpp:109
A featurizer maps activations x, (N, input_dim), to codes f = encode(x), (N, num_features),...
Definition featurizer.hpp:45
A tokenizer: added tokens are split out of the raw text first (longest match, earliest first); each r...
Definition text_tokenizer.hpp:166
Masked-language-model encoders: ESM-2 protein language models (PLM-2), Hugging Face EsmForMaskedLM ch...
The featurizer family's common interface (FEAT-1): models that rewrite activations as sparse codes ov...
Definition acquisition_functions.hpp:16
std::vector< int > SecondaryStructureLabels(const AnnotatedProtein &protein)
Three-state secondary structure per residue: 0 helix (Helix), 1 strand (Beta strand),...
std::vector< int > ConceptLabels(const AnnotatedProtein &protein, const std::string &type)
1 for each residue a feature of type covers, 0 elsewhere. A disulfide bond marks only its two cystein...
ResidueEmbeddings EmbedResidues(EncoderLM &model, const TextTokenizer &tokenizer, DeviceBackend *backend, const std::vector< std::string > &sequences, const std::vector< int64_t > &layers, int64_t max_tokens=1024)
Runs sequences through the model one at a time and keeps each residue's representation (not <cls> or ...
SparseCodes NeuronCodes(const std::vector< float > &rows, int64_t dim)
Dense values as codes, each dimension d twice: as feature 2d (its positive part) and 2d + 1 (its nega...
std::vector< float > SequenceWindowFeatures(const std::vector< std::string > &sequences, int64_t window)
The control a residue probe has to beat: each residue's local sequence, one-hot over the 20 amino aci...
std::vector< AnnotatedProtein > ReadAnnotatedProteins(const std::string &path)
ParseAnnotatedProteins on a file.
std::vector< std::pair< std::string, int64_t > > ConceptCounts(const std::vector< AnnotatedProtein > &proteins)
Each feature type, with the residues it covers over proteins, most first.
SparseCodes EncodeSparse(Featurizer &featurizer, const std::vector< float > &rows, DeviceBackend *backend, int64_t batch=4096)
featurizer's codes for every residue in rows (residues x input_dim), encoded in batches of batch and ...
std::vector< AnnotatedProtein > ParseAnnotatedProteins(std::string_view jsonl)
Reads JSON lines, one protein per line: {"accession", "sequence", "features": [{"type",...
ConceptMatch MatchConcept(const SparseCodes &codes, const std::vector< int > &labels, const std::string &concept_name, const std::vector< double > &thresholds={0.0, 0.15, 0.5, 0.6, 0.8})
InterPLM's matching: a feature predicts the concept on a residue when its value exceeds a threshold,...
Linear probes with controls (INT-3): a multi-class probe trainer, Hewitt and Liang's control task and...
Definition protein_concepts.hpp:36
std::string sequence
Definition protein_concepts.hpp:38
std::string accession
Definition protein_concepts.hpp:37
std::vector< ProteinFeature > features
Definition protein_concepts.hpp:39
The feature that best matches one concept.
Definition protein_concepts.hpp:135
std::string concept_name
Definition protein_concepts.hpp:136
int64_t features_above_half
Features whose best F1 for this concept is above 0.5.
Definition protein_concepts.hpp:144
double f1
Definition protein_concepts.hpp:142
double threshold
the share of the feature's largest value it must exceed
Definition protein_concepts.hpp:139
int64_t positives
residues with the concept
Definition protein_concepts.hpp:137
double precision
Definition protein_concepts.hpp:140
int64_t feature
Definition protein_concepts.hpp:138
double recall
Definition protein_concepts.hpp:141
One annotated stretch, numbered from 1, inclusive, as UniProt numbers it.
Definition protein_concepts.hpp:30
int64_t end
Definition protein_concepts.hpp:33
int64_t start
Definition protein_concepts.hpp:32
std::string type
UniProt's feature type: "Helix", "Binding site", "Disulfide bond", ...
Definition protein_concepts.hpp:31
Per-residue representations at some layers: for each layer, residues x hidden values,...
Definition protein_concepts.hpp:68
const float * row(size_t layer_index, int64_t residue) const
Definition protein_concepts.hpp:76
int64_t residues
Definition protein_concepts.hpp:71
std::vector< std::vector< float > > values
Definition protein_concepts.hpp:74
int64_t hidden
Definition protein_concepts.hpp:70
std::vector< int64_t > layers
Definition protein_concepts.hpp:69
std::vector< int64_t > offsets
Where each protein's residues start.
Definition protein_concepts.hpp:73
Sparse codes: for each residue, its active features and their values.
Definition protein_concepts.hpp:114
std::vector< float > value
Definition protein_concepts.hpp:118
int64_t residues() const
Definition protein_concepts.hpp:122
std::vector< float > max_value
Each feature's largest value.
Definition protein_concepts.hpp:120
std::vector< int32_t > feature
Definition protein_concepts.hpp:117
int64_t num_features
Definition protein_concepts.hpp:115
std::vector< int64_t > row_start
residues + 1 entries
Definition protein_concepts.hpp:116
The tokenizer interface (TOK-1): a pipeline of normalizer, pre-tokenizer, model, post-processor and d...