pulsatrix 1.1.0
Loading...
Searching...
No Matches
lr_scheduler.hpp File Reference

Learning-rate schedules (warmup, constant, linear, cosine) and a scheduler that applies one to an optimizer and its parameter groups (roadmap TRN-4). More...

#include <algorithm>
#include <cmath>
#include <cstdint>
#include <stdexcept>
#include <string>
#include <vector>
Include dependency graph for lr_scheduler.hpp:

Go to the source code of this file.

Classes

class  pulsatrix::LRSchedule
 A learning-rate multiplier as a function of the step, with the formulas of Hugging Face transformers' get_{constant,linear,cosine}_schedule_with_warmup. More...
 
class  pulsatrix::LRScheduler< Optimizer >
 Applies an LRSchedule to an optimizer (SGDOptimizer, AdamOptimizer, AdamWOptimizer): each rate becomes its base rate times the schedule's multiplier, like PyTorch's LambdaLR. More...
 

Namespaces

namespace  pulsatrix
 

Detailed Description

Learning-rate schedules (warmup, constant, linear, cosine) and a scheduler that applies one to an optimizer and its parameter groups (roadmap TRN-4).