pulsatrix 1.1.0
Loading...
Searching...
No Matches
pulsatrix::MetaspacePreTokenizer Member List

This is the complete list of members for pulsatrix::MetaspacePreTokenizer, including all inherited members.

MetaspacePreTokenizer(std::string replacement, PrependScheme scheme, bool split)pulsatrix::MetaspacePreTokenizer
pre_tokenize(std::vector< NormalizedString > &splits) const overridepulsatrix::MetaspacePreTokenizervirtual
~PreTokenizer()=defaultpulsatrix::PreTokenizervirtual