pulsatrix 1.1.0
Loading...
Searching...
No Matches
pulsatrix::LRScheduler< Optimizer > Class Template Reference

Applies an LRSchedule to an optimizer (SGDOptimizer, AdamOptimizer, AdamWOptimizer): each rate becomes its base rate times the schedule's multiplier, like PyTorch's LambdaLR. More...

#include <lr_scheduler.hpp>

Public Member Functions

 LRScheduler (Optimizer &optimizer, LRSchedule schedule)
 
void step ()
 Advances one step and sets the new rates.
 
int64_t last_step () const
 The step the current rates belong to.
 
void set_last_step (int64_t step)
 Jumps to step (e.g. when resuming from a checkpoint) and sets its rates.
 

Detailed Description

template<typename Optimizer>
class pulsatrix::LRScheduler< Optimizer >

Applies an LRSchedule to an optimizer (SGDOptimizer, AdamOptimizer, AdamWOptimizer): each rate becomes its base rate times the schedule's multiplier, like PyTorch's LambdaLR.

Note
The base rates are the optimizer's learning rate and each parameter group's (TRN-1) when the scheduler is built, so set up the groups first. Building the scheduler applies step 0 (with warmup, a rate of 0). Call step() once after each optimizer step.

Constructor & Destructor Documentation

◆ LRScheduler()

template<typename Optimizer >
pulsatrix::LRScheduler< Optimizer >::LRScheduler ( Optimizer &  optimizer,
LRSchedule  schedule 
)
inline

Member Function Documentation

◆ last_step()

template<typename Optimizer >
int64_t pulsatrix::LRScheduler< Optimizer >::last_step ( ) const
inline

The step the current rates belong to.

◆ set_last_step()

template<typename Optimizer >
void pulsatrix::LRScheduler< Optimizer >::set_last_step ( int64_t  step)
inline

Jumps to step (e.g. when resuming from a checkpoint) and sets its rates.

Exceptions
std::invalid_argumentif step < 0.

◆ step()

template<typename Optimizer >
void pulsatrix::LRScheduler< Optimizer >::step ( )
inline

Advances one step and sets the new rates.


The documentation for this class was generated from the following file: