|
pulsatrix 1.1.0
|
The plain transformer MLP, Linear -> activation -> Linear, of BERT, ESM and the original Transformer (PLM-1). The gated variant is SwiGLUModule.
More...
#include <optional>#include "pulsatrix/activation_module.hpp"#include "pulsatrix/linear_module.hpp"#include "pulsatrix/module.hpp"
Go to the source code of this file.
Classes | |
| class | pulsatrix::FeedForwardModule |
fc2(f(fc1(x))) over the last dimension. Shape (..., d_model) -> (..., d_model). More... | |
Namespaces | |
| namespace | pulsatrix |
The plain transformer MLP, Linear -> activation -> Linear, of BERT, ESM and the original Transformer (PLM-1). The gated variant is SwiGLUModule.