pulsatrix 1.1.0
Loading...
Searching...
No Matches
pulsatrix::Module Class Referenceabstract

Base class for every layer type (LinearModule, Conv2DModule, activations, ...). More...

#include <module.hpp>

Inheritance diagram for pulsatrix::Module:

Public Member Functions

virtual ~Module ()=default
 
Tensor forward (const Tensor &input)
 Runs this module's forward computation.
 
virtual std::optional< DeviceType > compute_device () const
 The device this module computes on, so forward() can reject an input on another device before any kernel sees it (roadmap FND-8).
 
std::pair< Tensor, NodeId > forward_traced (const Tensor &input, NodeId input_node, ComputationGraph &graph, Autograd &autograd)
 Runs forward() while also registering a ComputationGraph node (tagged with this module's op_type(), parented to input_node) and wiring an Autograd backward function that reuses this module's own backward() – the opt-in traced/explainable path, per Phase 2 Mission 0.
 
virtual Tensor propagate_relevance (const Tensor &relevance_out, const LRPRuleConfig &config)=0
 Computes this module's contribution to LRP relevance propagation.
 
virtual bool supports_lrp_rule (LRPRule rule) const
 Whether propagate_relevance() implements rule (no silent fallback: callers such as ExplainerContext::relevance_pass() throw rather than run a module on a rule it does not implement).
 
virtual Tensor backward (const Tensor &grad_output)=0
 Computes the gradient w.r.t. this module's input, given the gradient w.r.t. its output. Must be called after forward() – uses state cached from that call.
 
virtual OpType op_type () const =0
 This module's operation-category tag, for ComputationGraph node tagging.
 
virtual std::vector< NamedParamRef > named_parameters ()
 This module's trainable parameters, each with its hierarchical name – the one place a module declares its parameters (roadmap FND-1).
 
virtual std::vector< NamedBufferRef > named_buffers ()
 This module's buffers: state a checkpoint must save that no optimizer updates.
 
virtual std::vector< NamedModuleRef > named_children ()
 This module's direct sub-modules, each under the prefix its parameters carry in named_parameters(), in the same order (TRN-7: how ApplyLora() finds a model's Linear layers). Default: none. NamedModules() walks the whole tree.
 
virtual std::vector< ParamRef > parameters ()
 This module's trainable parameters and their gradients, for an optimizer to update uniformly across module types.
 
void set_requires_grad (bool requires_grad, const std::string &prefix="")
 Freezes (false) or unfreezes (true) parameters by name (roadmap FND-2).
 
virtual void set_training (bool training)
 Sets this module's training/eval mode. Defaults to training (matches every mainstream framework's Module default).
 
bool is_training () const
 Whether this module is currently in training mode.
 
virtual void release_activations ()
 Frees the tensors forward() kept for backward() and the relevance passes; containers release their children too. Inference that only needs the output calls it after each forward, so a deep model holds one layer's activations at a time (a 1024-token ESM-2 650M pass otherwise keeps over 10 GB of them).
 

Protected Member Functions

virtual Tensor forward_impl (const Tensor &input)=0
 The actual forward computation. Called by forward() after precondition checks.
 

Detailed Description

Base class for every layer type (LinearModule, Conv2DModule, activations, ...).

Note
forward()/forward_impl() is the NVI (non-virtual interface) idiom (oop_design/context_oop_design_patterns.md's Template Method section): the public entry point enforces preconditions every subclass gets for free; subclasses only implement the part that actually varies.
propagate_relevance is pure-virtual – charter non-negotiable #5. A module type without a defined LRP rule is a compile error, not a runtime "no default rule" exception (the Captum/Zennit failure mode this project exists to avoid).
A module does not build or own graph structure itself – ComputationGraph node registration and Autograd backward-function wiring are done generically by whichever caller opts into the traced path (Phase 2 Mission 0's Module::forward_traced), using backward()/op_type() below polymorphically. A module exposes plain tensor-in/tensor-out operations plus these two facts about itself; graph bookkeeping stays a separate concern (single responsibility).

Constructor & Destructor Documentation

◆ ~Module()

virtual pulsatrix::Module::~Module ( )
virtualdefault

Member Function Documentation

◆ backward()

virtual Tensor pulsatrix::Module::backward ( const Tensor &  grad_output)
pure virtual

Computes the gradient w.r.t. this module's input, given the gradient w.r.t. its output. Must be called after forward() – uses state cached from that call.

Parameters
grad_outputGradient w.r.t. this module's output.
Returns
Gradient w.r.t. this module's input.
Note
Promoted to the base class in Phase 2 Mission 0 – every existing subclass (LinearModule/ReluModule/Conv2DModule) already implemented this exact signature independently; making it virtual lets graph-wiring code (Module::forward_traced) call it polymorphically through a Module* without knowing the concrete subclass, the same way propagate_relevance already works.

Implemented in pulsatrix::ActivationModule, pulsatrix::AdaptiveAvgPool2DModule, pulsatrix::AggregatorModule, pulsatrix::AvgPool2DModule, pulsatrix::BatchNormModule, pulsatrix::BlockSparseFeaturizer, pulsatrix::CausalLM, pulsatrix::ConjunctionModule, pulsatrix::Conv2DModule, pulsatrix::Crosscoder, pulsatrix::DisjunctionModule, pulsatrix::DropoutModule, pulsatrix::EmbeddingModule, pulsatrix::EncoderBlock, pulsatrix::EncoderLM, pulsatrix::FeedForwardModule, pulsatrix::FlattenModule, pulsatrix::GroupNormModule, pulsatrix::GRUModule, pulsatrix::JumpReLUSparseAutoencoder, pulsatrix::LayerNormModule, pulsatrix::HiddenStateTap, pulsatrix::LinearModule, pulsatrix::LoReFTIntervention, pulsatrix::LSTMModule, pulsatrix::MambaModule, pulsatrix::MaxPool2DModule, pulsatrix::MultiHeadAttentionModule, pulsatrix::NegationModule, pulsatrix::ComponentLinear, pulsatrix::SequenceHead, pulsatrix::ReluModule, pulsatrix::ResidualModule, pulsatrix::RetNetModule, pulsatrix::RMSNormModule, pulsatrix::RNNModule, pulsatrix::RoPEModule, pulsatrix::RWKVModule, pulsatrix::SequentialModule, pulsatrix::SoftDecisionTree, pulsatrix::SoftmaxModule, pulsatrix::SparseAutoencoder, pulsatrix::SwiGLUModule, pulsatrix::TinyTagger, pulsatrix::TiedLMHeadModule, pulsatrix::TopKSparseAutoencoder, pulsatrix::Transcoder, pulsatrix::TransformerBlock, pulsatrix::TorchvisionResNet, and pulsatrix::TorchvisionVGG.

◆ compute_device()

virtual std::optional< DeviceType > pulsatrix::Module::compute_device ( ) const
inlinevirtual

The device this module computes on, so forward() can reject an input on another device before any kernel sees it (roadmap FND-8).

Returns
std::nullopt (the default) skips the check: a container whose layers check their own inputs, or a user module written before this existed. Every in-tree layer returns its device; EmbeddingModule doesn't, because it reads its indices through their own backend and so accepts them from any device.

Reimplemented in pulsatrix::ActivationModule, pulsatrix::AdaptiveAvgPool2DModule, pulsatrix::AggregatorModule, pulsatrix::AvgPool2DModule, pulsatrix::BatchNormModule, pulsatrix::BlockSparseFeaturizer, pulsatrix::CausalLM, pulsatrix::ConjunctionModule, pulsatrix::Conv2DModule, pulsatrix::Crosscoder, pulsatrix::DisjunctionModule, pulsatrix::DropoutModule, pulsatrix::EncoderBlock, pulsatrix::EncoderLM, pulsatrix::FeedForwardModule, pulsatrix::FlattenModule, pulsatrix::GroupNormModule, pulsatrix::GRUModule, pulsatrix::JumpReLUSparseAutoencoder, pulsatrix::LayerNormModule, pulsatrix::HiddenStateTap, pulsatrix::LinearModule, pulsatrix::LoReFTIntervention, pulsatrix::LSTMModule, pulsatrix::MambaModule, pulsatrix::MaxPool2DModule, pulsatrix::MultiHeadAttentionModule, pulsatrix::NegationModule, pulsatrix::ComponentLinear, pulsatrix::SequenceHead, pulsatrix::ReluModule, pulsatrix::ResidualModule, pulsatrix::RetNetModule, pulsatrix::RMSNormModule, pulsatrix::RNNModule, pulsatrix::RoPEModule, pulsatrix::RWKVModule, pulsatrix::SoftmaxModule, pulsatrix::SparseAutoencoder, pulsatrix::SwiGLUModule, pulsatrix::TiedLMHeadModule, pulsatrix::TopKSparseAutoencoder, pulsatrix::Transcoder, pulsatrix::TransformerBlock, pulsatrix::TorchvisionResNet, and pulsatrix::TorchvisionVGG.

◆ forward()

Tensor pulsatrix::Module::forward ( const Tensor &  input)
inline

Runs this module's forward computation.

Parameters
inputInput tensor. Must be non-empty.
Returns
The module's output.
Exceptions
std::invalid_argumentif input is empty – external boundary (campaign_exai_dl_library_adversarial_hardening.md, Mission 2, finding 15 systemic sweep): the single most external-facing check in the whole system, since every Module::forward() call – including from Phase 5's Python bindings – passes through this NVI wrapper first. Escalated from PULSATRIX_ASSERT-only.

◆ forward_impl()

virtual Tensor pulsatrix::Module::forward_impl ( const Tensor &  input)
protectedpure virtual

◆ forward_traced()

std::pair< Tensor, NodeId > pulsatrix::Module::forward_traced ( const Tensor &  input,
NodeId  input_node,
ComputationGraph &  graph,
Autograd &  autograd 
)
inline

Runs forward() while also registering a ComputationGraph node (tagged with this module's op_type(), parented to input_node) and wiring an Autograd backward function that reuses this module's own backward() – the opt-in traced/explainable path, per Phase 2 Mission 0.

Parameters
inputInput tensor. Must be non-empty (same precondition as forward()).
input_nodeId of the graph node producing input. Must already exist in graph.
graphGraph to add this module's output node to. Must outlive the returned node id's use (graph structure, per ComputationGraph's own design, persists past this call and past any subsequent backward pass).
autogradAutograd instance to register this node's backward function with.
Returns
The output tensor (identical to what forward(input) alone would return) and the new node's id, so a caller chaining multiple modules can thread node ids the same way it already threads tensors.
Note
Strictly additive: forward()/backward() are completely unaffected by this method's existence or use. Every graph-free call site (XorNetwork, every Phase 0/1 test) needs no changes.

◆ is_training()

bool pulsatrix::Module::is_training ( ) const
inline

Whether this module is currently in training mode.

◆ named_buffers()

virtual std::vector< NamedBufferRef > pulsatrix::Module::named_buffers ( )
inlinevirtual

This module's buffers: state a checkpoint must save that no optimizer updates.

Returns
{name, tensor} entries, named like named_parameters(), in a fixed order. Default: empty. BatchNormModule reports its running statistics; containers prefix their layers' buffers the way they prefix parameters.

Reimplemented in pulsatrix::BatchNormModule, pulsatrix::BlockSparseFeaturizer, pulsatrix::CausalLM, pulsatrix::Crosscoder, pulsatrix::EncoderBlock, pulsatrix::EncoderLM, pulsatrix::FeedForwardModule, pulsatrix::MultiHeadAttentionModule, pulsatrix::ResidualModule, pulsatrix::SequentialModule, pulsatrix::SwiGLUModule, pulsatrix::TinyTagger, pulsatrix::TopKSparseAutoencoder, pulsatrix::TransformerBlock, and pulsatrix::TorchvisionResNet.

◆ named_children()

virtual std::vector< NamedModuleRef > pulsatrix::Module::named_children ( )
inlinevirtual

This module's direct sub-modules, each under the prefix its parameters carry in named_parameters(), in the same order (TRN-7: how ApplyLora() finds a model's Linear layers). Default: none. NamedModules() walks the whole tree.

Note
The containers that build language models and their layers report their children: SequentialModule, ResidualModule, the attention, MLP and transformer blocks, CausalLM and EncoderLM. A module that doesn't is a leaf to NamedModules().

Reimplemented in pulsatrix::CausalLM, pulsatrix::EncoderBlock, pulsatrix::EncoderLM, pulsatrix::FeedForwardModule, pulsatrix::MultiHeadAttentionModule, pulsatrix::ResidualModule, pulsatrix::SequentialModule, pulsatrix::SoftDecisionTree, pulsatrix::SwiGLUModule, pulsatrix::TinyTagger, and pulsatrix::TransformerBlock.

◆ named_parameters()

virtual std::vector< NamedParamRef > pulsatrix::Module::named_parameters ( )
inlinevirtual

◆ op_type()

virtual OpType pulsatrix::Module::op_type ( ) const
pure virtual

This module's operation-category tag, for ComputationGraph node tagging.

Returns
This module's OpType (see op_type.hpp's closed set).
Note
Added in Phase 2 Mission 0 alongside backward() – lets graph-wiring code tag nodes generically through a Module* rather than switching on concrete subclass.

Implemented in pulsatrix::ActivationModule, pulsatrix::AdaptiveAvgPool2DModule, pulsatrix::AggregatorModule, pulsatrix::AvgPool2DModule, pulsatrix::BatchNormModule, pulsatrix::BlockSparseFeaturizer, pulsatrix::CausalLM, pulsatrix::ConjunctionModule, pulsatrix::Conv2DModule, pulsatrix::Crosscoder, pulsatrix::DisjunctionModule, pulsatrix::DropoutModule, pulsatrix::EmbeddingModule, pulsatrix::EncoderBlock, pulsatrix::EncoderLM, pulsatrix::FeedForwardModule, pulsatrix::FlattenModule, pulsatrix::GroupNormModule, pulsatrix::GRUModule, pulsatrix::JumpReLUSparseAutoencoder, pulsatrix::LayerNormModule, pulsatrix::HiddenStateTap, pulsatrix::LinearModule, pulsatrix::LoReFTIntervention, pulsatrix::LSTMModule, pulsatrix::MambaModule, pulsatrix::MaxPool2DModule, pulsatrix::MultiHeadAttentionModule, pulsatrix::NegationModule, pulsatrix::ComponentLinear, pulsatrix::SequenceHead, pulsatrix::ReluModule, pulsatrix::ResidualModule, pulsatrix::RetNetModule, pulsatrix::RMSNormModule, pulsatrix::RNNModule, pulsatrix::RoPEModule, pulsatrix::RWKVModule, pulsatrix::SequentialModule, pulsatrix::SoftDecisionTree, pulsatrix::SoftmaxModule, pulsatrix::SparseAutoencoder, pulsatrix::SwiGLUModule, pulsatrix::TinyTagger, pulsatrix::TiedLMHeadModule, pulsatrix::TopKSparseAutoencoder, pulsatrix::Transcoder, pulsatrix::TransformerBlock, pulsatrix::TorchvisionResNet, and pulsatrix::TorchvisionVGG.

◆ parameters()

virtual std::vector< ParamRef > pulsatrix::Module::parameters ( )
inlinevirtual

This module's trainable parameters and their gradients, for an optimizer to update uniformly across module types.

Returns
named_parameters() without the names – same tensors, same order.
Note
Not pure-virtual – unlike propagate_relevance, there is no charter non-negotiable requiring every module to define this; "no parameters" is a legitimate, common answer that shouldn't need restating per module type.
Still virtual only so subclasses written before named_parameters() existed keep compiling and training. Such a subclass reports no names, so name-keyed features can't see its parameters; new code overrides named_parameters() instead.

◆ propagate_relevance()

virtual Tensor pulsatrix::Module::propagate_relevance ( const Tensor &  relevance_out,
const LRPRuleConfig &  config 
)
pure virtual

Computes this module's contribution to LRP relevance propagation.

Parameters
relevance_outRelevance at this module's output.
configSelects the LRP rule variant.
Returns
Relevance at this module's input.

Implemented in pulsatrix::FlattenModule, pulsatrix::ActivationModule, pulsatrix::AdaptiveAvgPool2DModule, pulsatrix::AggregatorModule, pulsatrix::AvgPool2DModule, pulsatrix::BatchNormModule, pulsatrix::BlockSparseFeaturizer, pulsatrix::CausalLM, pulsatrix::ConjunctionModule, pulsatrix::Conv2DModule, pulsatrix::Crosscoder, pulsatrix::DisjunctionModule, pulsatrix::DropoutModule, pulsatrix::EmbeddingModule, pulsatrix::EncoderBlock, pulsatrix::EncoderLM, pulsatrix::FeedForwardModule, pulsatrix::GroupNormModule, pulsatrix::GRUModule, pulsatrix::JumpReLUSparseAutoencoder, pulsatrix::LayerNormModule, pulsatrix::HiddenStateTap, pulsatrix::LinearModule, pulsatrix::LoReFTIntervention, pulsatrix::LSTMModule, pulsatrix::MambaModule, pulsatrix::MaxPool2DModule, pulsatrix::MultiHeadAttentionModule, pulsatrix::NegationModule, pulsatrix::ComponentLinear, pulsatrix::SequenceHead, pulsatrix::ReluModule, pulsatrix::ResidualModule, pulsatrix::RetNetModule, pulsatrix::RMSNormModule, pulsatrix::RNNModule, pulsatrix::RoPEModule, pulsatrix::RWKVModule, pulsatrix::SequentialModule, pulsatrix::SoftDecisionTree, pulsatrix::SoftmaxModule, pulsatrix::SparseAutoencoder, pulsatrix::SwiGLUModule, pulsatrix::TinyTagger, pulsatrix::TiedLMHeadModule, pulsatrix::TopKSparseAutoencoder, pulsatrix::Transcoder, pulsatrix::TransformerBlock, pulsatrix::TorchvisionResNet, and pulsatrix::TorchvisionVGG.

◆ release_activations()

virtual void pulsatrix::Module::release_activations ( )
inlinevirtual

Frees the tensors forward() kept for backward() and the relevance passes; containers release their children too. Inference that only needs the output calls it after each forward, so a deep model holds one layer's activations at a time (a 1024-token ESM-2 650M pass otherwise keeps over 10 GB of them).

Note
Until the next forward(), backward() and relevance calls are invalid. The default has nothing to free.

Reimplemented in pulsatrix::ActivationModule, pulsatrix::BlockSparseFeaturizer, pulsatrix::Crosscoder, pulsatrix::EmbeddingModule, pulsatrix::EncoderBlock, pulsatrix::EncoderLM, pulsatrix::FeedForwardModule, pulsatrix::JumpReLUSparseAutoencoder, pulsatrix::LayerNormModule, pulsatrix::HiddenStateTap, pulsatrix::LinearModule, pulsatrix::LoReFTIntervention, pulsatrix::MultiHeadAttentionModule, pulsatrix::ComponentLinear, pulsatrix::SequenceHead, pulsatrix::RMSNormModule, pulsatrix::RoPEModule, pulsatrix::SoftmaxModule, pulsatrix::SparseAutoencoder, pulsatrix::SwiGLUModule, pulsatrix::TiedLMHeadModule, pulsatrix::TopKSparseAutoencoder, and pulsatrix::Transcoder.

◆ set_requires_grad()

void pulsatrix::Module::set_requires_grad ( bool  requires_grad,
const std::string &  prefix = "" 
)
inline

Freezes (false) or unfreezes (true) parameters by name (roadmap FND-2).

Parameters
requires_gradThe flag to set on every selected parameter's value tensor.
prefixEmpty selects every parameter. Otherwise selects the parameter named exactly prefix, and every parameter under it (mha.q_proj selects mha.q_proj.weight and mha.q_proj.bias, but mha.q selects nothing).
Exceptions
std::invalid_argumentif a non-empty prefix selects nothing – a mistyped name would otherwise silently leave the model trainable. Nothing is changed then.
Note
A frozen parameter's gradient is not accumulated by backward() and is not updated by an optimizer; the gradient w.r.t. the module's input is unchanged.

◆ set_training()

virtual void pulsatrix::Module::set_training ( bool  training)
inlinevirtual

Sets this module's training/eval mode. Defaults to training (matches every mainstream framework's Module default).

Note
Virtual since Phase 6 Mission 6 (SequentialModule) – Mission 5 originally shipped this as plain non-virtual state ("no composite container exists yet to cascade through"); SequentialModule overrides this to cascade to every contained layer, and needs virtual dispatch to do so correctly even when accessed through a Module* base pointer, not just its own concrete type.
BatchNormModule keeps running statistics in training mode and normalizes with them in eval mode; DropoutModule is the identity in eval mode. Call set_training(false) before explaining a model that has either, so a sample's explanation doesn't depend on the rest of its batch.

Reimplemented in pulsatrix::BlockSparseFeaturizer, pulsatrix::CausalLM, pulsatrix::Crosscoder, pulsatrix::EncoderBlock, pulsatrix::EncoderLM, pulsatrix::FeedForwardModule, pulsatrix::JumpReLUSparseAutoencoder, pulsatrix::MultiHeadAttentionModule, pulsatrix::SequenceHead, pulsatrix::ResidualModule, pulsatrix::SequentialModule, pulsatrix::SparseAutoencoder, pulsatrix::TinyTagger, pulsatrix::TopKSparseAutoencoder, pulsatrix::Transcoder, pulsatrix::TransformerBlock, pulsatrix::TorchvisionResNet, and pulsatrix::TorchvisionVGG.

◆ supports_lrp_rule()

virtual bool pulsatrix::Module::supports_lrp_rule ( LRPRule  rule) const
inlinevirtual

Whether propagate_relevance() implements rule (no silent fallback: callers such as ExplainerContext::relevance_pass() throw rather than run a module on a rule it does not implement).

Note
Default: only LRPRule::Epsilon, the rule every module implements. Overridden by LinearModule / Conv2DModule (all rules), by parameter-free pass-through modules whose relevance rule does not read the config at all, and by SequentialModule (all layers).

Reimplemented in pulsatrix::SequentialModule, pulsatrix::ActivationModule, pulsatrix::Conv2DModule, pulsatrix::DropoutModule, pulsatrix::FlattenModule, pulsatrix::HiddenStateTap, pulsatrix::LinearModule, pulsatrix::LoReFTIntervention, pulsatrix::MaxPool2DModule, pulsatrix::ReluModule, pulsatrix::SoftDecisionTree, and pulsatrix::TiedLMHeadModule.


The documentation for this class was generated from the following file: