The first difference on one line of the corpus.
More...
#include <tokenizer_parity.hpp>
|
| size_t | line = 0 |
| |
| std::string | field |
| | "ids", "plain_ids", "offsets", "decoded" or "decoded_skip".
|
| |
| std::string | text |
| |
| std::string | detail |
| |
The first difference on one line of the corpus.
◆ detail
| std::string pulsatrix::TokenizerMismatch::detail |
◆ field
| std::string pulsatrix::TokenizerMismatch::field |
"ids", "plain_ids", "offsets", "decoded" or "decoded_skip".
◆ line
| size_t pulsatrix::TokenizerMismatch::line = 0 |
◆ text
| std::string pulsatrix::TokenizerMismatch::text |
The documentation for this struct was generated from the following file: