|
pulsatrix 1.1.0
|
This is the complete list of members for pulsatrix::BpeModel, including all inherited members.
| BpeModel(std::unordered_map< std::string, int64_t > vocab, const std::vector< std::pair< std::string, std::string > > &merges, BpeOptions options={}) | pulsatrix::BpeModel | |
| id_to_token(int64_t id) const override | pulsatrix::BpeModel | virtual |
| token_to_id(std::string_view token) const override | pulsatrix::BpeModel | virtual |
| tokenize(std::string_view piece) const override | pulsatrix::BpeModel | virtual |
| vocab_size() const override | pulsatrix::BpeModel | inlinevirtual |
| ~TokenModel()=default | pulsatrix::TokenModel | virtual |