pulsatrix 1.1.0
Loading...
Searching...
No Matches
residual_module.hpp
Go to the documentation of this file.
1
6#pragma once
7
8#include <optional>
10
11namespace pulsatrix {
12
38class ResidualModule : public Module {
39public:
54
63
75 [[nodiscard]] Tensor backward(const Tensor& grad_output) override;
76
78 [[nodiscard]] OpType op_type() const override { return OpType::Elementwise; }
79
90 [[nodiscard]] Tensor propagate_relevance(const Tensor& relevance_out, const LRPRuleConfig& config) override;
91
94 [[nodiscard]] std::vector<NamedParamRef> named_parameters() override;
95 [[nodiscard]] std::vector<NamedModuleRef> named_children() override;
96
98 [[nodiscard]] std::vector<NamedBufferRef> named_buffers() override;
99
101 void set_training(bool training) override;
102
103 [[nodiscard]] Module& inner() { return *inner_; }
105 [[nodiscard]] Module* shortcut() { return shortcut_; }
106
107
109 [[nodiscard]] std::optional<DeviceType> compute_device() const override { return backend_->device(); }
110
111protected:
117 [[nodiscard]] Tensor forward_impl(const Tensor& input) override;
118
119private:
120 Module* inner_;
121 Module* shortcut_ = nullptr;
122 DeviceBackend* backend_;
123
124 Tensor last_x_;
125 Tensor last_f_x_;
126 bool has_forwarded_ = false;
127};
128
129} // namespace pulsatrix
Vendor-agnostic compute/memory backend. CPUBackend, CUDABackend (Phase 1.5), and HIPBackend (Phase 1....
Definition device_backend.hpp:248
virtual DeviceType device() const noexcept=0
Which device this backend's buffers reside on.
Base class for every layer type (LinearModule, Conv2DModule, activations, ...).
Definition module.hpp:78
y = x + inner->forward(x) for an arbitrary already-built Module. The classic ResNet shortcut connecti...
Definition residual_module.hpp:38
std::vector< NamedBufferRef > named_buffers() override
Every layer's named_buffers(), prefixed the same way as named_parameters().
std::vector< NamedParamRef > named_parameters() override
inner_'s own named_parameters(), prefixed inner., then the shortcut's, prefixed shortcut....
std::vector< NamedModuleRef > named_children() override
This module's direct sub-modules, each under the prefix its parameters carry in named_parameters(),...
void set_training(bool training) override
Cascades to inner_, the same way SequentialModule/MultiHeadAttentionModule do.
OpType op_type() const override
Elementwise per this module's own op_type() note above.
Definition residual_module.hpp:78
Module * shortcut()
The shortcut module, or nullptr for the identity.
Definition residual_module.hpp:105
Tensor backward(const Tensor &grad_output) override
Gradient w.r.t. this module's input: both paths receive grad_output unchanged (real gradient of a pla...
Module & inner()
Definition residual_module.hpp:103
Tensor forward_impl(const Tensor &input) override
y = x + inner_->forward(x).
ResidualModule(Module *inner, Module *shortcut, DeviceBackend *backend)
y = shortcut(x) + inner(x): a residual block whose shortcut is a module, such as ResNet's downsamplin...
std::optional< DeviceType > compute_device() const override
Where this layer computes, so forward() rejects an input on another device (FND-8).
Definition residual_module.hpp:109
Tensor propagate_relevance(const Tensor &relevance_out, const LRPRuleConfig &config) override
LRP relevance propagation: the residual epsilon/z-rule split between x and inner_->forward(x),...
ResidualModule(Module *inner, DeviceBackend *backend)
Constructs a residual wrapper around an existing Module.
N-dimensional tensor. Owns its data buffer exclusively; a DeviceBackend* is injected (not owned) – th...
Definition tensor.hpp:29
Abstract base every layer subclasses – NVI forward(), pure-virtual LRP contract.
Definition acquisition_functions.hpp:16
OpType
The op-type tag a Node carries. Charter Part 2 §3: nodes are tagged by a small closed set of op types...
Definition op_type.hpp:19
Configuration for LRP relevance propagation: which rule a module applies and its hyperparameters....
Definition lrp_rule_config.hpp:72