|
pulsatrix 1.1.0
|
Cross-entropy over a batch of tokens, with padding ignored and a caller-chosen normalizer for exact gradient accumulation (roadmap TRN-5). More...


Go to the source code of this file.
Classes | |
| class | pulsatrix::TokenCrossEntropyLoss |
| sum over tokens of -log softmax(logits)[target], divided by a normalizer; tokens whose target is the ignore index (padding) contribute nothing. More... | |
Namespaces | |
| namespace | pulsatrix |
Functions | |
| int64_t | pulsatrix::CountTargetTokens (const Tensor &targets, int64_t ignore_index=TokenCrossEntropyLoss::kIgnoreIndex) |
Number of entries of targets that aren't ignore_index: the normalizer for one accumulation window when summed over its micro-batches' targets. | |
Cross-entropy over a batch of tokens, with padding ignored and a caller-chosen normalizer for exact gradient accumulation (roadmap TRN-5).