pulsatrix 1.1.0
Loading...
Searching...
No Matches
pulsatrix::DistillationOptions Struct Reference

DistillationLoss's settings. More...

#include <distillation_loss.hpp>

Public Attributes

float temperature = 2.0f
 Softens both distributions; 1 compares them as they are. Must be positive.
 
float alpha = 0.5f
 The soft term's weight when there are targets; the hard term gets 1 − alpha. Without targets the loss is the soft term alone. Must be in [0, 1].
 
DistillationDivergence divergence = DistillationDivergence::Forward
 
int64_t ignore_index = TokenCrossEntropyLoss::kIgnoreIndex
 Rows whose target is this are left out of both terms (padding).
 

Detailed Description

DistillationLoss's settings.

Member Data Documentation

◆ alpha

float pulsatrix::DistillationOptions::alpha = 0.5f

The soft term's weight when there are targets; the hard term gets 1 − alpha. Without targets the loss is the soft term alone. Must be in [0, 1].

◆ divergence

DistillationDivergence pulsatrix::DistillationOptions::divergence = DistillationDivergence::Forward

◆ ignore_index

int64_t pulsatrix::DistillationOptions::ignore_index = TokenCrossEntropyLoss::kIgnoreIndex

Rows whose target is this are left out of both terms (padding).

◆ temperature

float pulsatrix::DistillationOptions::temperature = 2.0f

Softens both distributions; 1 compares them as they are. Must be positive.


The documentation for this struct was generated from the following file: