79 [[nodiscard]] int64_t
rank()
const {
return rank_; }
80 [[nodiscard]] int64_t
first()
const {
return first_; }
81 [[nodiscard]] int64_t
last()
const {
return last_; }
93 [[nodiscard]]
const std::vector<int64_t>&
last_positions()
const {
return last_rows_; }
104 Tensor r_, w_, b_, r_grad_, w_grad_, b_grad_;
105 std::vector<int64_t> last_rows_;
110 Tensor last_edit_relevance_;
A Llama-family decoder-only language model built from a Hugging Face config, and loading its Hugging ...
Token embedding, num_hidden_layers pre-norm transformer blocks, a final RMSNorm and a language-model ...
Definition causal_lm.hpp:82
Vendor-agnostic compute/memory backend. CPUBackend, CUDABackend (Phase 1.5), and HIPBackend (Phase 1....
Definition device_backend.hpp:248
virtual DeviceType device() const noexcept=0
Which device this backend's buffers reside on.
One block's LoReFT edit. Parameters, named as pyreft's state dict:
Definition loreft.hpp:52
int64_t rank() const
Definition loreft.hpp:79
void set_parameters(const std::vector< float > &rotation, const std::vector< float > &source_weight, const std::vector< float > &source_bias)
Replaces R^T (d, r), W^T (d, r) and b (r), row-major.
std::vector< NamedParamRef > named_parameters() override
This module's trainable parameters, each with its hierarchical name – the one place a module declares...
const Tensor & rotation() const
Definition loreft.hpp:82
void release_activations() override
Frees the tensors forward() kept for backward() and the relevance passes; containers release their ch...
const std::vector< int64_t > & last_positions() const
The positions the last forward edited, as row indices into (N * L).
Definition loreft.hpp:93
LoReFTIntervention(int64_t d_model, int64_t rank, int64_t first, int64_t last, DeviceBackend *backend, uint64_t seed)
R orthonormal (QR of a Gaussian matrix), and W and b as an nn.Linear(d, r) starts,...
Tensor backward(const Tensor &grad_output) override
The gradient with respect to the hidden states, accumulating R's, W's and b's.
void orthonormalize()
Makes R's rows orthonormal again (modified Gram-Schmidt in double, signs kept).
const Tensor & source_bias() const
Definition loreft.hpp:84
Tensor propagate_relevance(const Tensor &relevance_out, const LRPRuleConfig &config) override
Identity at the positions it doesn't edit. At the ones it does, the edit is the affine map h (I + (W^...
int64_t first() const
Definition loreft.hpp:80
OpType op_type() const override
This module's operation-category tag, for ComputationGraph node tagging.
Definition loreft.hpp:74
std::optional< DeviceType > compute_device() const override
The device this module computes on, so forward() can reject an input on another device before any ker...
Definition loreft.hpp:76
const Tensor & last_edit_relevance() const
The last relevance pass's part carried by the edit, (N, L, d), zero at unedited positions.
double orthonormality_error() const
The largest |R R^T - I| entry: 0 for an orthonormal R.
int64_t last() const
Definition loreft.hpp:81
const Tensor & source_weight() const
Definition loreft.hpp:83
Tensor forward_at(const Tensor &hidden, int64_t start, int64_t prompt_length) override
The edit for hidden states covering positions [start, start + L) of a sequence whose prompt is prompt...
bool supports_lrp_rule(LRPRule) const override
Whether propagate_relevance() implements rule (no silent fallback: callers such as ExplainerContext::...
Definition loreft.hpp:73
A trainable edit of the residual stream after a transformer block (TRN-9: LoReFT),...
Definition causal_lm.hpp:53
An N-dimensional shape. A plain aggregate of dimensions with no invariant beyond "non-negative dimens...
Definition shape.hpp:24
N-dimensional tensor. Owns its data buffer exclusively; a DeviceBackend* is injected (not owned) – th...
Definition tensor.hpp:29
Definition acquisition_functions.hpp:16
void SaveLoReFT(CausalLM &model, const std::string &path)
Saves the model's LoReFT interventions as safetensors: per layer i, pyreft's state dict under layer....
std::vector< LoReFTIntervention * > ApplyLoReFT(CausalLM &model, const LoReFTConfig &config={})
Installs a LoReFTIntervention after each of config.layers' blocks and freezes the rest of the model: ...
OpType
The op-type tag a Node carries. Charter Part 2 §3: nodes are tagged by a small closed set of op types...
Definition op_type.hpp:19
void OrthonormalizeLoReFT(CausalLM &model)
orthonormalize() on every LoReFT intervention of the model: call it after each optimizer step.
LRPRule
The LRP rule family a module applies. Semantics follow Zennit 1.0.0 exactly (Anders et al....
Definition lrp_rule_config.hpp:21
std::vector< LoReFTIntervention * > LoadLoReFT(CausalLM &model, const std::string &path)
Installs the interventions in a file SaveLoReFT() wrote, or one built from pyreft's per-intervention ...
Configuration for LRP relevance propagation: which rule a module applies and its hyperparameters....
Definition lrp_rule_config.hpp:72
Where and how large LoReFT's interventions are.
Definition loreft.hpp:28
int64_t last
Definition loreft.hpp:36
int64_t first
Edit the first first and the last last positions of each prompt: pyreft's "f<first>+l<last>"....
Definition loreft.hpp:35
uint64_t seed
Seeds R, W and b, the same on every platform.
Definition loreft.hpp:38
int64_t rank
r, the dimension of the edited subspace.
Definition loreft.hpp:30
std::vector< int64_t > layers
The blocks whose outputs are edited. Empty means every block.
Definition loreft.hpp:32