pulsatrix 1.1.0
Loading...
Searching...
No Matches
pulsatrix::CausalLM Class Reference

Token embedding, num_hidden_layers pre-norm transformer blocks, a final RMSNorm and a language-model head (tied to the embedding or its own bias-free Linear): the architecture of Llama, SmolLM2, Qwen2/2.5 and Qwen3. Maps (N, L) token ids to (N, L, vocab) logits. More...

#include <causal_lm.hpp>

Inheritance diagram for pulsatrix::CausalLM:
Collaboration diagram for pulsatrix::CausalLM:

Public Member Functions

 CausalLM (const HfModelConfig &config, DeviceBackend *backend, bool allow_unsupported=false)
 
Tensor backward (const Tensor &grad_output) override
 Computes the gradient w.r.t. this module's input, given the gradient w.r.t. its output. Must be called after forward() – uses state cached from that call.
 
Tensor propagate_relevance (const Tensor &relevance_out, const LRPRuleConfig &config) override
 Computes this module's contribution to LRP relevance propagation.
 
std::vector< Tensor > propagate_relevance_by_layer (const Tensor &relevance_out, const LRPRuleConfig &config)
 propagate_relevance, stopping at every block boundary: the relevance of the residual stream entering block 0 (the scaled embeddings), then leaving each block, each shaped like the hidden states (N, L, hidden_size). Summing the first over hidden_size gives propagate_relevance's per-token relevance.
 
OpType op_type () const override
 A chain of other modules, as SequentialModule.
 
std::vector< NamedParamRef > named_parameters () override
 This module's trainable parameters, each with its hierarchical name – the one place a module declares its parameters (roadmap FND-1).
 
std::vector< NamedModuleRef > named_children () override
 This module's direct sub-modules, each under the prefix its parameters carry in named_parameters(), in the same order (TRN-7: how ApplyLora() finds a model's Linear layers). Default: none. NamedModules() walks the whole tree.
 
std::vector< NamedBufferRef > named_buffers () override
 This module's buffers: state a checkpoint must save that no optimizer updates.
 
void set_training (bool training) override
 Sets this module's training/eval mode. Defaults to training (matches every mainstream framework's Module default).
 
std::optional< DeviceType > compute_device () const override
 The device this module computes on, so forward() can reject an input on another device before any kernel sees it (roadmap FND-8).
 
NextTokenLogitsFn next_token_logits (int64_t max_length)
 A cached next-token function for Generate (LLM-5): one pass over the prompt, then one position per new token.
 
void set_intervention (int64_t layer, std::unique_ptr< ResidualIntervention > intervention)
 Runs intervention on block layer's output, in forward(), backward(), the relevance passes and cached generation (TRN-9). Its parameters join named_parameters() as interventions.<layer>.*. nullptr removes it.
 
void set_interventions_enabled (bool enabled)
 false runs the model as if it had no interventions, everywhere they would run; true (the default) runs them again (TRN-11: the base model's relevance).
 
bool interventions_enabled () const
 
ResidualIntervention * intervention (int64_t layer)
 Block layer's intervention, or nullptr.
 
void set_hidden_state_hook (HiddenStateHook hook)
 Calls hook at every position during forward() (not during next_token_logits()'s cached passes). An empty function removes it.
 
const HfModelConfig & config () const
 
EmbeddingModule & embed_tokens ()
 
TransformerBlock & layer (int64_t i)
 
int64_t num_layers () const
 
RMSNormModule & norm ()
 
LinearModule * lm_head ()
 The untied language-model head, (hidden_size -> vocab); nullptr when tied to the embedding.
 
- Public Member Functions inherited from pulsatrix::Module
virtual ~Module ()=default
 
Tensor forward (const Tensor &input)
 Runs this module's forward computation.
 
std::pair< Tensor, NodeId > forward_traced (const Tensor &input, NodeId input_node, ComputationGraph &graph, Autograd &autograd)
 Runs forward() while also registering a ComputationGraph node (tagged with this module's op_type(), parented to input_node) and wiring an Autograd backward function that reuses this module's own backward() – the opt-in traced/explainable path, per Phase 2 Mission 0.
 
virtual bool supports_lrp_rule (LRPRule rule) const
 Whether propagate_relevance() implements rule (no silent fallback: callers such as ExplainerContext::relevance_pass() throw rather than run a module on a rule it does not implement).
 
virtual std::vector< ParamRef > parameters ()
 This module's trainable parameters and their gradients, for an optimizer to update uniformly across module types.
 
void set_requires_grad (bool requires_grad, const std::string &prefix="")
 Freezes (false) or unfreezes (true) parameters by name (roadmap FND-2).
 
bool is_training () const
 Whether this module is currently in training mode.
 
virtual void release_activations ()
 Frees the tensors forward() kept for backward() and the relevance passes; containers release their children too. Inference that only needs the output calls it after each forward, so a deep model holds one layer's activations at a time (a 1024-token ESM-2 650M pass otherwise keeps over 10 GB of them).
 

Protected Member Functions

Tensor forward_impl (const Tensor &input) override
 

Detailed Description

Token embedding, num_hidden_layers pre-norm transformer blocks, a final RMSNorm and a language-model head (tied to the embedding or its own bias-free Linear): the architecture of Llama, SmolLM2, Qwen2/2.5 and Qwen3. Maps (N, L) token ids to (N, L, vocab) logits.

Parameter names follow Hugging Face's with pulsatrix's module names inside a layer: embed_tokens.weight, layers.<i>.norm1.weight, layers.<i>.mha.q_proj.weight, ..., layers.<i>.swiglu.down_proj.weight, norm.weight, and lm_head.weight when untied. HuggingFaceMapping() lists how each Hugging Face tensor maps onto them.

Note
backward() and propagate_relevance() run through every layer, so the model trains and explains like any other module.

Constructor & Destructor Documentation

◆ CausalLM()

pulsatrix::CausalLM::CausalLM ( const HfModelConfig &  config,
DeviceBackend *  backend,
bool  allow_unsupported = false 
)
Parameters
configIts sizes, head counts, RoPE, biases, QK-Norm, epsilon and tying.
allow_unsupportedfalse (the default) refuses a config whose unsupported list isn't empty, since the model would run but compute something else.
Exceptions
std::invalid_argumentfor such a config, or invalid sizes.

Member Function Documentation

◆ backward()

Tensor pulsatrix::CausalLM::backward ( const Tensor &  grad_output)
overridevirtual

Computes the gradient w.r.t. this module's input, given the gradient w.r.t. its output. Must be called after forward() – uses state cached from that call.

Parameters
grad_outputGradient w.r.t. this module's output.
Returns
Gradient w.r.t. this module's input.
Note
Promoted to the base class in Phase 2 Mission 0 – every existing subclass (LinearModule/ReluModule/Conv2DModule) already implemented this exact signature independently; making it virtual lets graph-wiring code (Module::forward_traced) call it polymorphically through a Module* without knowing the concrete subclass, the same way propagate_relevance already works.

Implements pulsatrix::Module.

◆ compute_device()

std::optional< DeviceType > pulsatrix::CausalLM::compute_device ( ) const
inlineoverridevirtual

The device this module computes on, so forward() can reject an input on another device before any kernel sees it (roadmap FND-8).

Returns
std::nullopt (the default) skips the check: a container whose layers check their own inputs, or a user module written before this existed. Every in-tree layer returns its device; EmbeddingModule doesn't, because it reads its indices through their own backend and so accepts them from any device.

Reimplemented from pulsatrix::Module.

◆ config()

const HfModelConfig & pulsatrix::CausalLM::config ( ) const
inline

◆ embed_tokens()

EmbeddingModule & pulsatrix::CausalLM::embed_tokens ( )
inline

◆ forward_impl()

Tensor pulsatrix::CausalLM::forward_impl ( const Tensor &  input)
overrideprotectedvirtual
Exceptions
std::invalid_argumentif input isn't (N, L) token ids in the vocabulary.

Implements pulsatrix::Module.

◆ intervention()

ResidualIntervention * pulsatrix::CausalLM::intervention ( int64_t  layer)
inline

Block layer's intervention, or nullptr.

Exceptions
std::out_of_range.

◆ interventions_enabled()

bool pulsatrix::CausalLM::interventions_enabled ( ) const
inline

◆ layer()

TransformerBlock & pulsatrix::CausalLM::layer ( int64_t  i)
inline

◆ lm_head()

LinearModule * pulsatrix::CausalLM::lm_head ( )
inline

The untied language-model head, (hidden_size -> vocab); nullptr when tied to the embedding.

◆ named_buffers()

std::vector< NamedBufferRef > pulsatrix::CausalLM::named_buffers ( )
overridevirtual

This module's buffers: state a checkpoint must save that no optimizer updates.

Returns
{name, tensor} entries, named like named_parameters(), in a fixed order. Default: empty. BatchNormModule reports its running statistics; containers prefix their layers' buffers the way they prefix parameters.

Reimplemented from pulsatrix::Module.

◆ named_children()

std::vector< NamedModuleRef > pulsatrix::CausalLM::named_children ( )
overridevirtual

This module's direct sub-modules, each under the prefix its parameters carry in named_parameters(), in the same order (TRN-7: how ApplyLora() finds a model's Linear layers). Default: none. NamedModules() walks the whole tree.

Note
The containers that build language models and their layers report their children: SequentialModule, ResidualModule, the attention, MLP and transformer blocks, CausalLM and EncoderLM. A module that doesn't is a leaf to NamedModules().

Reimplemented from pulsatrix::Module.

◆ named_parameters()

std::vector< NamedParamRef > pulsatrix::CausalLM::named_parameters ( )
overridevirtual

This module's trainable parameters, each with its hierarchical name – the one place a module declares its parameters (roadmap FND-1).

Returns
{name, {value, grad}} entries pointing directly at this module's own members, in a fixed order. Names are unique within the module tree. Default: empty (a parameterless module like ReluModule needs no override).
Note
Override this, not parameters(): saving, loading, freezing by name and optimizer parameter groups all key on these names.

Reimplemented from pulsatrix::Module.

◆ next_token_logits()

NextTokenLogitsFn pulsatrix::CausalLM::next_token_logits ( int64_t  max_length)

A cached next-token function for Generate (LLM-5): one pass over the prompt, then one position per new token.

Note
Holds pointers into this model, which must outlive it.
Exceptions
std::logic_errorfor a model with bidirectional attention (config().causal false).

◆ norm()

RMSNormModule & pulsatrix::CausalLM::norm ( )
inline

◆ num_layers()

int64_t pulsatrix::CausalLM::num_layers ( ) const
inline

◆ op_type()

OpType pulsatrix::CausalLM::op_type ( ) const
inlineoverridevirtual

A chain of other modules, as SequentialModule.

Implements pulsatrix::Module.

◆ propagate_relevance()

Tensor pulsatrix::CausalLM::propagate_relevance ( const Tensor &  relevance_out,
const LRPRuleConfig &  config 
)
overridevirtual

Computes this module's contribution to LRP relevance propagation.

Parameters
relevance_outRelevance at this module's output.
configSelects the LRP rule variant.
Returns
Relevance at this module's input.

Implements pulsatrix::Module.

◆ propagate_relevance_by_layer()

std::vector< Tensor > pulsatrix::CausalLM::propagate_relevance_by_layer ( const Tensor &  relevance_out,
const LRPRuleConfig &  config 
)

propagate_relevance, stopping at every block boundary: the relevance of the residual stream entering block 0 (the scaled embeddings), then leaving each block, each shaped like the hidden states (N, L, hidden_size). Summing the first over hidden_size gives propagate_relevance's per-token relevance.

Exceptions
std::logic_errorbefore any forward().

◆ set_hidden_state_hook()

void pulsatrix::CausalLM::set_hidden_state_hook ( HiddenStateHook  hook)
inline

Calls hook at every position during forward() (not during next_token_logits()'s cached passes). An empty function removes it.

◆ set_intervention()

void pulsatrix::CausalLM::set_intervention ( int64_t  layer,
std::unique_ptr< ResidualIntervention >  intervention 
)

Runs intervention on block layer's output, in forward(), backward(), the relevance passes and cached generation (TRN-9). Its parameters join named_parameters() as interventions.<layer>.*. nullptr removes it.

Exceptions
std::out_of_rangefor a layer index the model doesn't have.

◆ set_interventions_enabled()

void pulsatrix::CausalLM::set_interventions_enabled ( bool  enabled)
inline

false runs the model as if it had no interventions, everywhere they would run; true (the default) runs them again (TRN-11: the base model's relevance).

◆ set_training()

void pulsatrix::CausalLM::set_training ( bool  training)
overridevirtual

Sets this module's training/eval mode. Defaults to training (matches every mainstream framework's Module default).

Note
Virtual since Phase 6 Mission 6 (SequentialModule) – Mission 5 originally shipped this as plain non-virtual state ("no composite container exists yet to cascade through"); SequentialModule overrides this to cascade to every contained layer, and needs virtual dispatch to do so correctly even when accessed through a Module* base pointer, not just its own concrete type.
BatchNormModule keeps running statistics in training mode and normalizes with them in eval mode; DropoutModule is the identity in eval mode. Call set_training(false) before explaining a model that has either, so a sample's explanation doesn't depend on the rest of its batch.

Reimplemented from pulsatrix::Module.


The documentation for this class was generated from the following file: