pulsatrix 1.1.0
Loading...
Searching...
No Matches
pulsatrix::TemplatePostProcessor::Item Struct Reference

One template item: the sequence itself (special empty), or special tokens. More...

#include <byte_level_bpe.hpp>

Public Member Functions

bool is_sequence () const
 

Public Attributes

std::vector< int64_t > special_ids
 
std::vector< std::string > special_tokens
 

Detailed Description

One template item: the sequence itself (special empty), or special tokens.

Member Function Documentation

◆ is_sequence()

bool pulsatrix::TemplatePostProcessor::Item::is_sequence ( ) const
inline

Member Data Documentation

◆ special_ids

std::vector<int64_t> pulsatrix::TemplatePostProcessor::Item::special_ids

◆ special_tokens

std::vector<std::string> pulsatrix::TemplatePostProcessor::Item::special_tokens

The documentation for this struct was generated from the following file: