|
pulsatrix 1.1.0
|
Reads or replaces a transformer block's MLP output during forward() (FEAT-5), for collecting a transcoder's training pairs and splicing it in. More...


Go to the source code of this file.
Namespaces | |
| namespace | pulsatrix |
| namespace | pulsatrix::detail |
Typedefs | |
| using | pulsatrix::MlpHook = std::function< Tensor(const Tensor &mlp_input, const Tensor &mlp_output)> |
| Called with an MLP's input (after the block's norm, where it has one) and its output (the branch added to the residual stream), both shaped like the block's input. What it returns, the same shape, is added to the residual stream instead of the output: return the output unchanged to only read it. | |
Functions | |
| Tensor | pulsatrix::detail::ApplyMlpHook (const MlpHook &hook, const Tensor &input, const Tensor &output) |
Runs hook and checks its result keeps the shape. | |
Reads or replaces a transformer block's MLP output during forward() (FEAT-5), for collecting a transcoder's training pairs and splicing it in.