pulsatrix 1.1.0
Loading...
Searching...
No Matches
pulsatrix::PostProcessor Class Referenceabstract

Adds tokens around an encoding (BOS and EOS, sentence pairs, ...). More...

#include <text_tokenizer.hpp>

Inheritance diagram for pulsatrix::PostProcessor:

Public Member Functions

virtual ~PostProcessor ()=default
 
virtual Encoding process (Encoding encoding, bool add_special_tokens) const =0
 

Detailed Description

Adds tokens around an encoding (BOS and EOS, sentence pairs, ...).

Constructor & Destructor Documentation

◆ ~PostProcessor()

virtual pulsatrix::PostProcessor::~PostProcessor ( )
virtualdefault

Member Function Documentation

◆ process()

virtual Encoding pulsatrix::PostProcessor::process ( Encoding  encoding,
bool  add_special_tokens 
) const
pure virtual

The documentation for this class was generated from the following file: