pulsatrix 1.1.0
Loading...
Searching...
No Matches
feed_forward_module.hpp File Reference

The plain transformer MLP, Linear -> activation -> Linear, of BERT, ESM and the original Transformer (PLM-1). The gated variant is SwiGLUModule. More...

#include <optional>
#include "pulsatrix/activation_module.hpp"
#include "pulsatrix/linear_module.hpp"
#include "pulsatrix/module.hpp"
Include dependency graph for feed_forward_module.hpp:

Go to the source code of this file.

Classes

class  pulsatrix::FeedForwardModule
 fc2(f(fc1(x))) over the last dimension. Shape (..., d_model) -> (..., d_model). More...
 

Namespaces

namespace  pulsatrix
 

Detailed Description

The plain transformer MLP, Linear -> activation -> Linear, of BERT, ESM and the original Transformer (PLM-1). The gated variant is SwiGLUModule.