|
pulsatrix 1.1.0
|
Adam optimizer – operates uniformly across any Module's parameters(). More...
#include <stdexcept>#include <unordered_map>#include "pulsatrix/module.hpp"#include "pulsatrix/param_groups.hpp"

Go to the source code of this file.
Classes | |
| class | pulsatrix::AdamOptimizer |
| Adam (Kingma & Ba, 2015): per-parameter moving averages of gradient (m) and squared gradient (v), with bias correction. More... | |
| struct | pulsatrix::AdamOptimizer::AdamState |
| One parameter's Adam state: first and second moments, and its step count. More... | |
| class | pulsatrix::AdamWOptimizer |
| AdamW (Loshchilov & Hutter, "Decoupled Weight Decay Regularization", arXiv 1711.05101): Adam whose weight decay shrinks the weights directly, w <- (1 - lr * wd) * w, before the Adam step, instead of adding wd * w to the gradient (roadmap TRN-2). More... | |
Namespaces | |
| namespace | pulsatrix |
Adam optimizer – operates uniformly across any Module's parameters().