|
pulsatrix 1.1.0
|
Concatenates the tokens (Hugging Face's Fuse decoder). More...
#include <tokenizer_components.hpp>


Public Member Functions | |
| std::string | decode (const std::vector< std::string > &tokens) const override |
| The decoded text: decode_chain() joined. | |
Public Member Functions inherited from pulsatrix::Decoder | |
| virtual | ~Decoder ()=default |
| virtual std::vector< std::string > | decode_chain (std::vector< std::string > tokens) const |
| The tokens after this decoder: by default decode()'s text as one token. | |
Concatenates the tokens (Hugging Face's Fuse decoder).
|
overridevirtual |
The decoded text: decode_chain() joined.
Reimplemented from pulsatrix::Decoder.