Applies an LRSchedule to an optimizer (SGDOptimizer, AdamOptimizer, AdamWOptimizer): each rate becomes its base rate times the schedule's multiplier, like PyTorch's LambdaLR.
More...
#include <lr_scheduler.hpp>
|
| | LRScheduler (Optimizer &optimizer, LRSchedule schedule) |
| |
| void | step () |
| | Advances one step and sets the new rates.
|
| |
| int64_t | last_step () const |
| | The step the current rates belong to.
|
| |
| void | set_last_step (int64_t step) |
| | Jumps to step (e.g. when resuming from a checkpoint) and sets its rates.
|
| |
template<typename Optimizer>
class pulsatrix::LRScheduler< Optimizer >
Applies an LRSchedule to an optimizer (SGDOptimizer, AdamOptimizer, AdamWOptimizer): each rate becomes its base rate times the schedule's multiplier, like PyTorch's LambdaLR.
- Note
- The base rates are the optimizer's learning rate and each parameter group's (TRN-1) when the scheduler is built, so set up the groups first. Building the scheduler applies step 0 (with warmup, a rate of 0). Call step() once after each optimizer step.
◆ LRScheduler()
template<typename Optimizer >
◆ last_step()
template<typename Optimizer >
The step the current rates belong to.
◆ set_last_step()
template<typename Optimizer >
Jumps to step (e.g. when resuming from a checkpoint) and sets its rates.
- Exceptions
-
| std::invalid_argument | if step < 0. |
◆ step()
template<typename Optimizer >
Advances one step and sets the new rates.
The documentation for this class was generated from the following file: