pulsatrix 1.1.0
Loading...
Searching...
No Matches
rope_module.hpp
Go to the documentation of this file.
1
5#pragma once
6
7#include <optional>
8#include <vector>
10
11namespace pulsatrix {
12
18enum class RoPELayout {
24};
25
50class RoPEModule : public Module {
51public:
65 RoPEModule(int64_t head_dim, DeviceBackend* backend, float base = 10000.0f,
67
75 RoPEModule(int64_t head_dim, DeviceBackend* backend, std::vector<double> inverse_frequencies,
77
84 void set_position_offset(int64_t offset);
85 [[nodiscard]] int64_t position_offset() const { return position_offset_; }
86 [[nodiscard]] RoPELayout layout() const { return layout_; }
87 [[nodiscard]] float base() const { return base_; }
88
105 [[nodiscard]] Tensor backward(const Tensor& grad_output) override;
106
122 [[nodiscard]] OpType op_type() const override { return OpType::Elementwise; }
123
155 [[nodiscard]] Tensor propagate_relevance(const Tensor& relevance_out, const LRPRuleConfig& config) override;
156
157
159 [[nodiscard]] std::optional<DeviceType> compute_device() const override { return backend_->device(); }
160
161 void release_activations() override;
162
163protected:
171 [[nodiscard]] Tensor forward_impl(const Tensor& input) override;
172
173private:
174 int64_t head_dim_;
175 float base_;
176 RoPELayout layout_;
177 std::vector<double> inv_freq_;
178 int64_t position_offset_ = 0;
179 DeviceBackend* backend_;
180 Tensor last_input_;
181 Tensor last_output_;
182 bool has_forwarded_ = false;
183
192 void ensure_tables(int64_t seq_len, int64_t offset);
193 Tensor cos_table_;
194 Tensor sin_table_;
195 int64_t table_seq_len_ = 0;
196 int64_t table_offset_ = 0;
197 // Offset the cached forward used, so backward and LRP rotate by the same angles even if
198 // set_position_offset is called in between.
199 int64_t last_offset_ = 0;
200};
201
202} // namespace pulsatrix
Vendor-agnostic compute/memory backend. CPUBackend, CUDABackend (Phase 1.5), and HIPBackend (Phase 1....
Definition device_backend.hpp:248
virtual DeviceType device() const noexcept=0
Which device this backend's buffers reside on.
Base class for every layer type (LinearModule, Conv2DModule, activations, ...).
Definition module.hpp:78
Rotary Position Embedding (RoPE, Su et al. 2021): a fixed, non-learnable, position-dependent rotation...
Definition rope_module.hpp:50
void release_activations() override
Frees the tensors forward() kept for backward() and the relevance passes; containers release their ch...
Tensor forward_impl(const Tensor &input) override
Applies the per-position pair rotation.
std::optional< DeviceType > compute_device() const override
Where this layer computes, so forward() rejects an input on another device (FND-8).
Definition rope_module.hpp:159
RoPEModule(int64_t head_dim, DeviceBackend *backend, std::vector< double > inverse_frequencies, RoPELayout layout=RoPELayout::AdjacentPairs)
RoPE with explicit per-pair frequencies (LLM-6): pair i turns by pos * inverse_frequencies[i] instead...
float base() const
Definition rope_module.hpp:87
OpType op_type() const override
Elementwise per the charter's closed OpType set.
Definition rope_module.hpp:122
RoPEModule(int64_t head_dim, DeviceBackend *backend, float base=10000.0f, RoPELayout layout=RoPELayout::AdjacentPairs)
Constructs a RoPE module.
Tensor backward(const Tensor &grad_output) override
Computes the gradient w.r.t. this module's input.
void set_position_offset(int64_t offset)
Sets the position of the first row of the L axis, so forward() rotates rows offset,...
Tensor propagate_relevance(const Tensor &relevance_out, const LRPRuleConfig &config) override
Standard weighted-connection epsilon-rule LRP for this fixed linear map.
int64_t position_offset() const
Definition rope_module.hpp:85
RoPELayout layout() const
Definition rope_module.hpp:86
N-dimensional tensor. Owns its data buffer exclusively; a DeviceBackend* is injected (not owned) – th...
Definition tensor.hpp:29
Abstract base every layer subclasses – NVI forward(), pure-virtual LRP contract.
Definition acquisition_functions.hpp:16
RoPELayout
Which features RoPE rotates together (LLM-1).
Definition rope_module.hpp:18
OpType
The op-type tag a Node carries. Charter Part 2 §3: nodes are tagged by a small closed set of op types...
Definition op_type.hpp:19
Configuration for LRP relevance propagation: which rule a module applies and its hyperparameters....
Definition lrp_rule_config.hpp:72