|
| | EncoderBlock (const AttentionConfig &attention, int64_t d_ff, DeviceBackend *backend, const EncoderBlockOptions &options={}) |
| |
| Tensor | backward (const Tensor &grad_output) override |
| |
| Tensor | propagate_relevance (const Tensor &relevance_out, const LRPRuleConfig &config) override |
| |
| OpType | op_type () const override |
| | Owns the residual adds, as TransformerBlock does.
|
| |
| std::vector< NamedParamRef > | named_parameters () override |
| | This module's trainable parameters, each with its hierarchical name – the one place a module declares its parameters (roadmap FND-1).
|
| |
| std::vector< NamedModuleRef > | named_children () override |
| | This module's direct sub-modules, each under the prefix its parameters carry in named_parameters(), in the same order (TRN-7: how ApplyLora() finds a model's Linear layers). Default: none. NamedModules() walks the whole tree.
|
| |
| std::vector< NamedBufferRef > | named_buffers () override |
| | This module's buffers: state a checkpoint must save that no optimizer updates.
|
| |
| void | set_training (bool training) override |
| | Sets this module's training/eval mode. Defaults to training (matches every mainstream framework's Module default).
|
| |
| std::optional< DeviceType > | compute_device () const override |
| | The device this module computes on, so forward() can reject an input on another device before any kernel sees it (roadmap FND-8).
|
| |
| int64_t | d_model () const |
| |
| const EncoderBlockOptions & | options () const |
| |
| void | release_activations () override |
| | Frees the tensors forward() kept for backward() and the relevance passes; containers release their children too. Inference that only needs the output calls it after each forward, so a deep model holds one layer's activations at a time (a 1024-token ESM-2 650M pass otherwise keeps over 10 GB of them).
|
| |
|
| Module & | norm1 () |
| |
| MultiHeadAttentionModule & | mha () |
| |
| Module & | norm2 () |
| |
| Module & | mlp () |
| |
| void | set_mlp_hook (MlpHook hook) |
| | Calls hook on the MLP's input and output during forward() (MlpHook). An empty function removes it.
|
| |
| virtual | ~Module ()=default |
| |
| Tensor | forward (const Tensor &input) |
| | Runs this module's forward computation.
|
| |
| std::pair< Tensor, NodeId > | forward_traced (const Tensor &input, NodeId input_node, ComputationGraph &graph, Autograd &autograd) |
| | Runs forward() while also registering a ComputationGraph node (tagged with this module's op_type(), parented to input_node) and wiring an Autograd backward function that reuses this module's own backward() – the opt-in traced/explainable path, per Phase 2 Mission 0.
|
| |
| virtual bool | supports_lrp_rule (LRPRule rule) const |
| | Whether propagate_relevance() implements rule (no silent fallback: callers such as ExplainerContext::relevance_pass() throw rather than run a module on a rule it does not implement).
|
| |
| virtual std::vector< ParamRef > | parameters () |
| | This module's trainable parameters and their gradients, for an optimizer to update uniformly across module types.
|
| |
| void | set_requires_grad (bool requires_grad, const std::string &prefix="") |
| | Freezes (false) or unfreezes (true) parameters by name (roadmap FND-2).
|
| |
| bool | is_training () const |
| | Whether this module is currently in training mode.
|
| |
One encoder layer. Shape (N, L, d_model) -> (N, L, d_model).
- Pre-norm:
y1 = x + attn(norm1(x)), y2 = y1 + mlp(norm2(y1)).
- Post-norm:
y1 = norm1(x + attn(x)), y2 = norm2(y1 + mlp(y1)).
Attention is configured in full by AttentionConfig. An encoder leaves causal off; ESM-2 uses rotate-half RoPE with biases.
Parameters are named norm1.*, mha.*, norm2.* and mlp.*. A plain MLP's are mlp.fc1.* and mlp.fc2.*; a gated MLP's are mlp.gate_proj.*, mlp.up_proj.* and mlp.down_proj.*.
- Note
- LRP follows TransformerBlock's rules:
- each residual add is split by the two-term epsilon rule;
- norms and activations pass relevance through unchanged, as AttnLRP prescribes;
- attention and linear layers use their own rules. Pre-norm with RMSNorm and a gated MLP gives the same output, gradient and relevance as TransformerBlock; the tests check this.