50 [[nodiscard]] int64_t
num_tokens()
const {
return num_tokens_; }
54 [[noreturn]]
void ThrowForBadTarget(
const Tensor& targets, int64_t classes)
const;
57 int64_t ignore_index_;
62 float normalizer_ = 1.0f;
63 int64_t num_tokens_ = 0;
64 bool has_forwarded_ =
false;
Vendor-agnostic compute/memory backend. CPUBackend, CUDABackend (Phase 1.5), and HIPBackend (Phase 1....
Definition device_backend.hpp:248
An N-dimensional shape. A plain aggregate of dimensions with no invariant beyond "non-negative dimens...
Definition shape.hpp:24
N-dimensional tensor. Owns its data buffer exclusively; a DeviceBackend* is injected (not owned) – th...
Definition tensor.hpp:29
sum over tokens of -log softmax(logits)[target], divided by a normalizer; tokens whose target is the ...
Definition token_cross_entropy_loss.hpp:28
static constexpr int64_t kIgnoreIndex
PyTorch's default ignore_index.
Definition token_cross_entropy_loss.hpp:31
TokenCrossEntropyLoss(DeviceBackend *backend, int64_t ignore_index=kIgnoreIndex)
float forward(const Tensor &logits, const Tensor &targets, float normalizer=0.0f)
Computes the loss and caches what backward() needs.
Tensor backward() const
Gradient w.r.t. the logits, same shape.
int64_t num_tokens() const
Real (not ignored) tokens in the last forward().
Definition token_cross_entropy_loss.hpp:50
Definition acquisition_functions.hpp:16
int64_t CountTargetTokens(const Tensor &targets, int64_t ignore_index=TokenCrossEntropyLoss::kIgnoreIndex)
Number of entries of targets that aren't ignore_index: the normalizer for one accumulation window whe...
N-dimensional tensor – owns a buffer via DeviceBackend*, RAII (Rule of Five).