pulsatrix 1.1.0
Loading...
Searching...
No Matches
pulsatrix::LoReFTIntervention Class Reference

One block's LoReFT edit. Parameters, named as pyreft's state dict: More...

#include <loreft.hpp>

Inheritance diagram for pulsatrix::LoReFTIntervention:
Collaboration diagram for pulsatrix::LoReFTIntervention:

Public Member Functions

 LoReFTIntervention (int64_t d_model, int64_t rank, int64_t first, int64_t last, DeviceBackend *backend, uint64_t seed)
 R orthonormal (QR of a Gaussian matrix), and W and b as an nn.Linear(d, r) starts, uniform on ±1/sqrt(d), as pyreft initializes them. Unlike LoRA, the edit isn't zero at the start.
 
Tensor forward_at (const Tensor &hidden, int64_t start, int64_t prompt_length) override
 The edit for hidden states covering positions [start, start + L) of a sequence whose prompt is prompt_length positions long. forward(x) is forward_at(x, 0, L). Cached generation calls it once on the prompt and then once per new token, each a later start.
 
Tensor backward (const Tensor &grad_output) override
 The gradient with respect to the hidden states, accumulating R's, W's and b's.
 
Tensor propagate_relevance (const Tensor &relevance_out, const LRPRuleConfig &config) override
 Identity at the positions it doesn't edit. At the ones it does, the edit is the affine map h (I + (W^T - R^T) R) + b R, and the rule runs on its two paths together, the identity and the edit, as for a split LoRA layer (TRN-8). The edit's part is kept in last_edit_relevance(). Every rule is supported; under epsilon the result is the relevance of the map as a whole.
 
bool supports_lrp_rule (LRPRule) const override
 Whether propagate_relevance() implements rule (no silent fallback: callers such as ExplainerContext::relevance_pass() throw rather than run a module on a rule it does not implement).
 
OpType op_type () const override
 This module's operation-category tag, for ComputationGraph node tagging.
 
std::vector< NamedParamRef > named_parameters () override
 This module's trainable parameters, each with its hierarchical name – the one place a module declares its parameters (roadmap FND-1).
 
std::optional< DeviceType > compute_device () const override
 The device this module computes on, so forward() can reject an input on another device before any kernel sees it (roadmap FND-8).
 
void release_activations () override
 Frees the tensors forward() kept for backward() and the relevance passes; containers release their children too. Inference that only needs the output calls it after each forward, so a deep model holds one layer's activations at a time (a 1024-token ESM-2 650M pass otherwise keeps over 10 GB of them).
 
int64_t rank () const
 
int64_t first () const
 
int64_t last () const
 
const Tensor & rotation () const
 
const Tensor & source_weight () const
 
const Tensor & source_bias () const
 
void set_parameters (const std::vector< float > &rotation, const std::vector< float > &source_weight, const std::vector< float > &source_bias)
 Replaces R^T (d, r), W^T (d, r) and b (r), row-major.
 
void orthonormalize ()
 Makes R's rows orthonormal again (modified Gram-Schmidt in double, signs kept).
 
double orthonormality_error () const
 The largest |R R^T - I| entry: 0 for an orthonormal R.
 
const std::vector< int64_t > & last_positions () const
 The positions the last forward edited, as row indices into (N * L).
 
const Tensor & last_edit_relevance () const
 The last relevance pass's part carried by the edit, (N, L, d), zero at unedited positions.
 
- Public Member Functions inherited from pulsatrix::Module
virtual ~Module ()=default
 
Tensor forward (const Tensor &input)
 Runs this module's forward computation.
 
std::pair< Tensor, NodeId > forward_traced (const Tensor &input, NodeId input_node, ComputationGraph &graph, Autograd &autograd)
 Runs forward() while also registering a ComputationGraph node (tagged with this module's op_type(), parented to input_node) and wiring an Autograd backward function that reuses this module's own backward() – the opt-in traced/explainable path, per Phase 2 Mission 0.
 
virtual std::vector< NamedBufferRef > named_buffers ()
 This module's buffers: state a checkpoint must save that no optimizer updates.
 
virtual std::vector< NamedModuleRef > named_children ()
 This module's direct sub-modules, each under the prefix its parameters carry in named_parameters(), in the same order (TRN-7: how ApplyLora() finds a model's Linear layers). Default: none. NamedModules() walks the whole tree.
 
virtual std::vector< ParamRef > parameters ()
 This module's trainable parameters and their gradients, for an optimizer to update uniformly across module types.
 
void set_requires_grad (bool requires_grad, const std::string &prefix="")
 Freezes (false) or unfreezes (true) parameters by name (roadmap FND-2).
 
virtual void set_training (bool training)
 Sets this module's training/eval mode. Defaults to training (matches every mainstream framework's Module default).
 
bool is_training () const
 Whether this module is currently in training mode.
 

Additional Inherited Members

- Protected Member Functions inherited from pulsatrix::ResidualIntervention
Tensor forward_impl (const Tensor &hidden) override
 The actual forward computation. Called by forward() after precondition checks.
 

Detailed Description

One block's LoReFT edit. Parameters, named as pyreft's state dict:

  • rotate_layer: R^T, (d, r) with orthonormal columns (pyreft's layout);
  • learned_source.weight: W^T, (d, r), the transpose of pyreft's (r, d), like every pulsatrix Linear weight;
  • learned_source.bias: b, (r).
    Note
    Gradient steps move R off the orthonormal set; call orthonormalize() after each optimizer step (OrthonormalizeLoReFT() does it for a model). pyreft keeps R orthonormal with torch's orthogonal parametrization instead, so training paths differ, but the edit, its gradient with respect to R and its relevance are the same for the same R.

Constructor & Destructor Documentation

◆ LoReFTIntervention()

pulsatrix::LoReFTIntervention::LoReFTIntervention ( int64_t  d_model,
int64_t  rank,
int64_t  first,
int64_t  last,
DeviceBackend *  backend,
uint64_t  seed 
)

R orthonormal (QR of a Gaussian matrix), and W and b as an nn.Linear(d, r) starts, uniform on ±1/sqrt(d), as pyreft initializes them. Unlike LoRA, the edit isn't zero at the start.

Exceptions
std::invalid_argumentunless 1 <= rank <= d and first, last >= 0.

Member Function Documentation

◆ backward()

Tensor pulsatrix::LoReFTIntervention::backward ( const Tensor &  grad_output)
overridevirtual

The gradient with respect to the hidden states, accumulating R's, W's and b's.

Implements pulsatrix::Module.

◆ compute_device()

std::optional< DeviceType > pulsatrix::LoReFTIntervention::compute_device ( ) const
inlineoverridevirtual

The device this module computes on, so forward() can reject an input on another device before any kernel sees it (roadmap FND-8).

Returns
std::nullopt (the default) skips the check: a container whose layers check their own inputs, or a user module written before this existed. Every in-tree layer returns its device; EmbeddingModule doesn't, because it reads its indices through their own backend and so accepts them from any device.

Reimplemented from pulsatrix::Module.

◆ first()

int64_t pulsatrix::LoReFTIntervention::first ( ) const
inline

◆ forward_at()

Tensor pulsatrix::LoReFTIntervention::forward_at ( const Tensor &  hidden,
int64_t  start,
int64_t  prompt_length 
)
overridevirtual

The edit for hidden states covering positions [start, start + L) of a sequence whose prompt is prompt_length positions long. forward(x) is forward_at(x, 0, L). Cached generation calls it once on the prompt and then once per new token, each a later start.

Implements pulsatrix::ResidualIntervention.

◆ last()

int64_t pulsatrix::LoReFTIntervention::last ( ) const
inline

◆ last_edit_relevance()

const Tensor & pulsatrix::LoReFTIntervention::last_edit_relevance ( ) const

The last relevance pass's part carried by the edit, (N, L, d), zero at unedited positions.

Exceptions
std::logic_errorbefore a relevance pass, or after a new forward().

◆ last_positions()

const std::vector< int64_t > & pulsatrix::LoReFTIntervention::last_positions ( ) const
inline

The positions the last forward edited, as row indices into (N * L).

◆ named_parameters()

std::vector< NamedParamRef > pulsatrix::LoReFTIntervention::named_parameters ( )
overridevirtual

This module's trainable parameters, each with its hierarchical name – the one place a module declares its parameters (roadmap FND-1).

Returns
{name, {value, grad}} entries pointing directly at this module's own members, in a fixed order. Names are unique within the module tree. Default: empty (a parameterless module like ReluModule needs no override).
Note
Override this, not parameters(): saving, loading, freezing by name and optimizer parameter groups all key on these names.

Reimplemented from pulsatrix::Module.

◆ op_type()

OpType pulsatrix::LoReFTIntervention::op_type ( ) const
inlineoverridevirtual

This module's operation-category tag, for ComputationGraph node tagging.

Returns
This module's OpType (see op_type.hpp's closed set).
Note
Added in Phase 2 Mission 0 alongside backward() – lets graph-wiring code tag nodes generically through a Module* rather than switching on concrete subclass.

Implements pulsatrix::Module.

◆ orthonormality_error()

double pulsatrix::LoReFTIntervention::orthonormality_error ( ) const

The largest |R R^T - I| entry: 0 for an orthonormal R.

◆ orthonormalize()

void pulsatrix::LoReFTIntervention::orthonormalize ( )

Makes R's rows orthonormal again (modified Gram-Schmidt in double, signs kept).

◆ propagate_relevance()

Tensor pulsatrix::LoReFTIntervention::propagate_relevance ( const Tensor &  relevance_out,
const LRPRuleConfig &  config 
)
overridevirtual

Identity at the positions it doesn't edit. At the ones it does, the edit is the affine map h (I + (W^T - R^T) R) + b R, and the rule runs on its two paths together, the identity and the edit, as for a split LoRA layer (TRN-8). The edit's part is kept in last_edit_relevance(). Every rule is supported; under epsilon the result is the relevance of the map as a whole.

Implements pulsatrix::Module.

◆ rank()

int64_t pulsatrix::LoReFTIntervention::rank ( ) const
inline

◆ release_activations()

void pulsatrix::LoReFTIntervention::release_activations ( )
overridevirtual

Frees the tensors forward() kept for backward() and the relevance passes; containers release their children too. Inference that only needs the output calls it after each forward, so a deep model holds one layer's activations at a time (a 1024-token ESM-2 650M pass otherwise keeps over 10 GB of them).

Note
Until the next forward(), backward() and relevance calls are invalid. The default has nothing to free.

Reimplemented from pulsatrix::Module.

◆ rotation()

const Tensor & pulsatrix::LoReFTIntervention::rotation ( ) const
inline

◆ set_parameters()

void pulsatrix::LoReFTIntervention::set_parameters ( const std::vector< float > &  rotation,
const std::vector< float > &  source_weight,
const std::vector< float > &  source_bias 
)

Replaces R^T (d, r), W^T (d, r) and b (r), row-major.

Exceptions
std::invalid_argumenton a wrong size.

◆ source_bias()

const Tensor & pulsatrix::LoReFTIntervention::source_bias ( ) const
inline

◆ source_weight()

const Tensor & pulsatrix::LoReFTIntervention::source_weight ( ) const
inline

◆ supports_lrp_rule()

bool pulsatrix::LoReFTIntervention::supports_lrp_rule ( LRPRule  rule) const
inlineoverridevirtual

Whether propagate_relevance() implements rule (no silent fallback: callers such as ExplainerContext::relevance_pass() throw rather than run a module on a rule it does not implement).

Note
Default: only LRPRule::Epsilon, the rule every module implements. Overridden by LinearModule / Conv2DModule (all rules), by parameter-free pass-through modules whose relevance rule does not read the config at all, and by SequentialModule (all layers).

Reimplemented from pulsatrix::Module.


The documentation for this class was generated from the following file: