|
| | ResidualModule (Module *inner, DeviceBackend *backend) |
| | Constructs a residual wrapper around an existing Module.
|
| |
| | ResidualModule (Module *inner, Module *shortcut, DeviceBackend *backend) |
| | y = shortcut(x) + inner(x): a residual block whose shortcut is a module, such as ResNet's downsampling 1x1 conv + BatchNorm (KS-9). Relevance is split between the two branches in proportion to their contributions, then each passes its share back.
|
| |
| Tensor | backward (const Tensor &grad_output) override |
| | Gradient w.r.t. this module's input: both paths receive grad_output unchanged (real gradient of a plain sum), then inner_'s own backward() adds its contribution.
|
| |
| OpType | op_type () const override |
| | Elementwise per this module's own op_type() note above.
|
| |
| Tensor | propagate_relevance (const Tensor &relevance_out, const LRPRuleConfig &config) override |
| | LRP relevance propagation: the residual epsilon/z-rule split between x and inner_->forward(x), then inner_'s own propagate_relevance() for its share.
|
| |
| std::vector< NamedParamRef > | named_parameters () override |
| | inner_'s own named_parameters(), prefixed inner., then the shortcut's, prefixed shortcut. – this module owns none of its own.
|
| |
| std::vector< NamedModuleRef > | named_children () override |
| | This module's direct sub-modules, each under the prefix its parameters carry in named_parameters(), in the same order (TRN-7: how ApplyLora() finds a model's Linear layers). Default: none. NamedModules() walks the whole tree.
|
| |
| std::vector< NamedBufferRef > | named_buffers () override |
| | Every layer's named_buffers(), prefixed the same way as named_parameters().
|
| |
| void | set_training (bool training) override |
| | Cascades to inner_, the same way SequentialModule/MultiHeadAttentionModule do.
|
| |
| Module & | inner () |
| |
| Module * | shortcut () |
| | The shortcut module, or nullptr for the identity.
|
| |
| std::optional< DeviceType > | compute_device () const override |
| | Where this layer computes, so forward() rejects an input on another device (FND-8).
|
| |
| virtual | ~Module ()=default |
| |
| Tensor | forward (const Tensor &input) |
| | Runs this module's forward computation.
|
| |
| std::pair< Tensor, NodeId > | forward_traced (const Tensor &input, NodeId input_node, ComputationGraph &graph, Autograd &autograd) |
| | Runs forward() while also registering a ComputationGraph node (tagged with this module's op_type(), parented to input_node) and wiring an Autograd backward function that reuses this module's own backward() – the opt-in traced/explainable path, per Phase 2 Mission 0.
|
| |
| virtual bool | supports_lrp_rule (LRPRule rule) const |
| | Whether propagate_relevance() implements rule (no silent fallback: callers such as ExplainerContext::relevance_pass() throw rather than run a module on a rule it does not implement).
|
| |
| virtual std::vector< ParamRef > | parameters () |
| | This module's trainable parameters and their gradients, for an optimizer to update uniformly across module types.
|
| |
| void | set_requires_grad (bool requires_grad, const std::string &prefix="") |
| | Freezes (false) or unfreezes (true) parameters by name (roadmap FND-2).
|
| |
| bool | is_training () const |
| | Whether this module is currently in training mode.
|
| |
| virtual void | release_activations () |
| | Frees the tensors forward() kept for backward() and the relevance passes; containers release their children too. Inference that only needs the output calls it after each forward, so a deep model holds one layer's activations at a time (a 1024-token ESM-2 650M pass otherwise keeps over 10 GB of them).
|
| |
y = x + inner->forward(x) for an arbitrary already-built Module. The classic ResNet shortcut connection, owning its own native relevance-split rule – the charter's explicitly named failure mode to avoid is Captum/Zennit's "Canonizer
surgery" (an external post-hoc graph rewrite for residual connections).
Not a new LRP rule: reuses this codebase's own two-term weighted-sum epsilon/z-rule (LSTMModule's c_t = f_t*c_{t-1}+i_t*g_t, GRUModule's analogous carry-split, TransformerBlock's own two residual adds), weight fixed at 1 – resolved at Phase 3 activation, not re-derived here.
- Note
inner_ is held by non-owning pointer – "not owned; must outlive this object", the same convention as SequentialModule's layers_. Any already-composed Module can be wrapped: a SequentialModule chaining Conv2DModule/ BatchNormModule/ReluModule for a real ResNet basic block, a bare Conv2DModule, a LinearModule, even another ResidualModule.
-
op_type() reuses OpType::Elementwise – this module's entire purpose is the residual add itself (fixed weight 1, computed independently per element), the same category TransformerBlock's own residual adds already use. inner_'s own op_type() is unaffected.
-
inner_->forward(x) must return the same shape as x – this module cannot validate that structurally in advance (it does not know inner_'s internals); a mismatch surfaces as Tensor's own shape-mismatch error at the add or at the relevance split.