pulsatrix 1.1.0
Loading...
Searching...
No Matches
adam_optimizer.hpp File Reference

Adam optimizer – operates uniformly across any Module's parameters(). More...

#include <stdexcept>
#include <unordered_map>
#include "pulsatrix/module.hpp"
#include "pulsatrix/param_groups.hpp"
Include dependency graph for adam_optimizer.hpp:
This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Classes

class  pulsatrix::AdamOptimizer
 Adam (Kingma & Ba, 2015): per-parameter moving averages of gradient (m) and squared gradient (v), with bias correction. More...
 
struct  pulsatrix::AdamOptimizer::AdamState
 One parameter's Adam state: first and second moments, and its step count. More...
 
class  pulsatrix::AdamWOptimizer
 AdamW (Loshchilov & Hutter, "Decoupled Weight Decay Regularization", arXiv 1711.05101): Adam whose weight decay shrinks the weights directly, w <- (1 - lr * wd) * w, before the Adam step, instead of adding wd * w to the gradient (roadmap TRN-2). More...
 

Namespaces

namespace  pulsatrix
 

Detailed Description

Adam optimizer – operates uniformly across any Module's parameters().