One template item: the sequence itself (special empty), or special tokens.
More...
#include <byte_level_bpe.hpp>
One template item: the sequence itself (special empty), or special tokens.
◆ is_sequence()
| bool pulsatrix::TemplatePostProcessor::Item::is_sequence |
( |
| ) |
const |
|
inline |
◆ special_ids
| std::vector<int64_t> pulsatrix::TemplatePostProcessor::Item::special_ids |
◆ special_tokens
| std::vector<std::string> pulsatrix::TemplatePostProcessor::Item::special_tokens |
The documentation for this struct was generated from the following file: