|
pulsatrix 1.1.0
|
This is the complete list of members for pulsatrix::TokenModel, including all inherited members.
| id_to_token(int64_t id) const =0 | pulsatrix::TokenModel | pure virtual |
| token_to_id(std::string_view token) const =0 | pulsatrix::TokenModel | pure virtual |
| tokenize(std::string_view piece) const =0 | pulsatrix::TokenModel | pure virtual |
| vocab_size() const =0 | pulsatrix::TokenModel | pure virtual |
| ~TokenModel()=default | pulsatrix::TokenModel | virtual |