pulsatrix 1.1.0
Loading...
Searching...
No Matches
grad_clipping.hpp File Reference

Global gradient-norm clipping (roadmap TRN-3). More...

Include dependency graph for grad_clipping.hpp:

Go to the source code of this file.

Namespaces

namespace  pulsatrix
 

Functions

float pulsatrix::ClipGradNorm (Module &module, float max_norm, bool error_if_nonfinite=false)
 Scales every trainable parameter's gradient so their global L2 norm is at most max_norm, as torch.nn.utils.clip_grad_norm_ does, and returns the norm before clipping.
 

Detailed Description

Global gradient-norm clipping (roadmap TRN-3).