|
pulsatrix 1.1.0
|
DistillationLoss's settings. More...
#include <distillation_loss.hpp>
Public Attributes | |
| float | temperature = 2.0f |
| Softens both distributions; 1 compares them as they are. Must be positive. | |
| float | alpha = 0.5f |
| The soft term's weight when there are targets; the hard term gets 1 − alpha. Without targets the loss is the soft term alone. Must be in [0, 1]. | |
| DistillationDivergence | divergence = DistillationDivergence::Forward |
| int64_t | ignore_index = TokenCrossEntropyLoss::kIgnoreIndex |
| Rows whose target is this are left out of both terms (padding). | |
DistillationLoss's settings.
| float pulsatrix::DistillationOptions::alpha = 0.5f |
The soft term's weight when there are targets; the hard term gets 1 − alpha. Without targets the loss is the soft term alone. Must be in [0, 1].
| DistillationDivergence pulsatrix::DistillationOptions::divergence = DistillationDivergence::Forward |
| int64_t pulsatrix::DistillationOptions::ignore_index = TokenCrossEntropyLoss::kIgnoreIndex |
Rows whose target is this are left out of both terms (padding).
| float pulsatrix::DistillationOptions::temperature = 2.0f |
Softens both distributions; 1 compares them as they are. Must be positive.