pulsatrix 1.1.0
Loading...
Searching...
No Matches
protein_sequences.hpp
Go to the documentation of this file.
1
5#pragma once
6
7#include <string>
8#include <string_view>
9#include <vector>
10
12
13namespace pulsatrix {
14
29[[nodiscard]] TextTokenizer MakeEsmTokenizer(const std::vector<std::string>& vocab);
31[[nodiscard]] TextTokenizer LoadEsmTokenizer(const std::string& vocab_path);
32
36 std::string id;
38 std::string description;
41 std::string sequence;
42};
43
49[[nodiscard]] std::vector<FastaRecord> ParseFasta(std::string_view text);
51[[nodiscard]] std::vector<FastaRecord> ReadFasta(const std::string& path);
53[[nodiscard]] std::string WriteFasta(const std::vector<FastaRecord>& records, size_t width = 60);
54
55} // namespace pulsatrix
A tokenizer: added tokens are split out of the raw text first (longest match, earliest first); each r...
Definition text_tokenizer.hpp:166
Definition acquisition_functions.hpp:16
std::string WriteFasta(const std::vector< FastaRecord > &records, size_t width=60)
FASTA text for records, sequences wrapped at width residues (0: one line).
std::vector< FastaRecord > ParseFasta(std::string_view text)
Parses FASTA text: records start with >, sequences may span lines, and blank lines and ; comment line...
TextTokenizer MakeEsmTokenizer(const std::vector< std::string > &vocab)
ESM's tokenizer (Hugging Face's EsmTokenizer) over a vocabulary listed one token per line,...
TextTokenizer LoadEsmTokenizer(const std::string &vocab_path)
MakeEsmTokenizer on a vocab.txt.
std::vector< FastaRecord > ReadFasta(const std::string &path)
ParseFasta on a file.
One FASTA record.
Definition protein_sequences.hpp:34
std::string description
The rest of the header line, after the first run of whitespace.
Definition protein_sequences.hpp:38
std::string id
The header's first word, without the >.
Definition protein_sequences.hpp:36
std::string sequence
The sequence lines joined, with whitespace removed. Case and characters are kept as written,...
Definition protein_sequences.hpp:41
The tokenizer interface (TOK-1): a pipeline of normalizer, pre-tokenizer, model, post-processor and d...