|
pulsatrix 1.1.0
|
Learning-rate schedules (warmup, constant, linear, cosine) and a scheduler that applies one to an optimizer and its parameter groups (roadmap TRN-4). More...
#include <algorithm>#include <cmath>#include <cstdint>#include <stdexcept>#include <string>#include <vector>
Go to the source code of this file.
Classes | |
| class | pulsatrix::LRSchedule |
| A learning-rate multiplier as a function of the step, with the formulas of Hugging Face transformers' get_{constant,linear,cosine}_schedule_with_warmup. More... | |
| class | pulsatrix::LRScheduler< Optimizer > |
| Applies an LRSchedule to an optimizer (SGDOptimizer, AdamOptimizer, AdamWOptimizer): each rate becomes its base rate times the schedule's multiplier, like PyTorch's LambdaLR. More... | |
Namespaces | |
| namespace | pulsatrix |
Learning-rate schedules (warmup, constant, linear, cosine) and a scheduler that applies one to an optimizer and its parameter groups (roadmap TRN-4).