42 Tensor out = hook(position, x);
43 if (out.
shape() != x.
shape())
throw std::invalid_argument(
"hidden-state hook: the replacement must keep the hidden states' shape");
64 if (hidden.
rank() != 3)
throw std::invalid_argument(
"ResidualIntervention::forward: hidden states must be (N, L, d)");
139 [[nodiscard]] int64_t
num_layers()
const {
return static_cast<int64_t
>(layers_.size()); }
151 [[nodiscard]]
Tensor ScaleEmbeddings(
const Tensor& x)
const;
155 std::vector<std::unique_ptr<TransformerBlock>> layers_;
157 std::unique_ptr<TiedLMHeadModule> tied_head_;
158 std::unique_ptr<LinearModule> lm_head_;
161 std::vector<std::unique_ptr<ResidualIntervention>> interventions_;
162 bool interventions_enabled_ =
true;
165 return interventions_enabled_ ? interventions_[i].get() :
nullptr;
Token embedding, num_hidden_layers pre-norm transformer blocks, a final RMSNorm and a language-model ...
Definition causal_lm.hpp:82
std::vector< Tensor > propagate_relevance_by_layer(const Tensor &relevance_out, const LRPRuleConfig &config)
propagate_relevance, stopping at every block boundary: the relevance of the residual stream entering ...
TransformerBlock & layer(int64_t i)
Definition causal_lm.hpp:138
int64_t num_layers() const
Definition causal_lm.hpp:139
std::vector< NamedParamRef > named_parameters() override
This module's trainable parameters, each with its hierarchical name – the one place a module declares...
std::vector< NamedBufferRef > named_buffers() override
This module's buffers: state a checkpoint must save that no optimizer updates.
void set_intervention(int64_t layer, std::unique_ptr< ResidualIntervention > intervention)
Runs intervention on block layer's output, in forward(), backward(), the relevance passes and cached ...
void set_hidden_state_hook(HiddenStateHook hook)
Calls hook at every position during forward() (not during next_token_logits()'s cached passes)....
Definition causal_lm.hpp:134
LinearModule * lm_head()
The untied language-model head, (hidden_size -> vocab); nullptr when tied to the embedding.
Definition causal_lm.hpp:143
void set_interventions_enabled(bool enabled)
false runs the model as if it had no interventions, everywhere they would run; true (the default) run...
Definition causal_lm.hpp:127
Tensor backward(const Tensor &grad_output) override
Computes the gradient w.r.t. this module's input, given the gradient w.r.t. its output....
std::vector< NamedModuleRef > named_children() override
This module's direct sub-modules, each under the prefix its parameters carry in named_parameters(),...
const HfModelConfig & config() const
Definition causal_lm.hpp:136
OpType op_type() const override
A chain of other modules, as SequentialModule.
Definition causal_lm.hpp:103
void set_training(bool training) override
Sets this module's training/eval mode. Defaults to training (matches every mainstream framework's Mod...
ResidualIntervention * intervention(int64_t layer)
Block layer's intervention, or nullptr.
Definition causal_lm.hpp:130
Tensor forward_impl(const Tensor &input) override
Tensor propagate_relevance(const Tensor &relevance_out, const LRPRuleConfig &config) override
Computes this module's contribution to LRP relevance propagation.
CausalLM(const HfModelConfig &config, DeviceBackend *backend, bool allow_unsupported=false)
bool interventions_enabled() const
Definition causal_lm.hpp:128
RMSNormModule & norm()
Definition causal_lm.hpp:140
NextTokenLogitsFn next_token_logits(int64_t max_length)
A cached next-token function for Generate (LLM-5): one pass over the prompt, then one position per ne...
std::optional< DeviceType > compute_device() const override
The device this module computes on, so forward() can reject an input on another device before any ker...
Definition causal_lm.hpp:108
EmbeddingModule & embed_tokens()
Definition causal_lm.hpp:137
Vendor-agnostic compute/memory backend. CPUBackend, CUDABackend (Phase 1.5), and HIPBackend (Phase 1....
Definition device_backend.hpp:248
virtual DeviceType device() const noexcept=0
Which device this backend's buffers reside on.
Embedding lookup table, rank-2 input (N, L) of float-encoded indices -> rank-3 output (N,...
Definition embedding_module.hpp:39
A Hugging Face checkpoint directory's weights: model.safetensors, or the shards listed by model....
Definition hf_model.hpp:139
y = x @ W + b, batched (x is (N, in_features), y is (N, out_features)) – migrated from the original u...
Definition linear_module.hpp:43
Base class for every layer type (LinearModule, Conv2DModule, activations, ...).
Definition module.hpp:78
y_{n,i} = gamma_i * x_{n,i} / rms(x_n), rms(x_n) = sqrt(mean_i(x_{n,i}^2) + eps), computed independen...
Definition rms_norm_module.hpp:32
A trainable edit of the residual stream after a transformer block (TRN-9: LoReFT),...
Definition causal_lm.hpp:53
Tensor forward_impl(const Tensor &hidden) override
The actual forward computation. Called by forward() after precondition checks.
Definition causal_lm.hpp:63
virtual Tensor forward_at(const Tensor &hidden, int64_t start, int64_t prompt_length)=0
The edit for hidden states covering positions [start, start + L) of a sequence whose prompt is prompt...
An N-dimensional shape. A plain aggregate of dimensions with no invariant beyond "non-negative dimens...
Definition shape.hpp:24
int64_t dim(size_t index) const
Size of a single dimension.
Definition shape.hpp:93
N-dimensional tensor. Owns its data buffer exclusively; a DeviceBackend* is injected (not owned) – th...
Definition tensor.hpp:29
int64_t rank() const
Number of dimensions – shape().rank().
Definition tensor.hpp:119
const Shape & shape() const
This tensor's shape.
Definition tensor.hpp:113
Lookup-table (row-select) layer – y = W[index], batched (N, L) -> (N, L, embedding_dim).
Text generation from a language model: greedy decoding and seeded sampling with temperature,...
Hugging Face checkpoints: config.json and single or sharded safetensors (IO-5).
Dense/fully-connected layer – the reference Module implementation.
Abstract base every layer subclasses – NVI forward(), pure-virtual LRP contract.
Tensor ApplyHiddenStateHook(const HiddenStateHook &hook, int64_t position, const Tensor &x)
Runs hook and checks its result keeps the shape.
Definition causal_lm.hpp:41
Definition acquisition_functions.hpp:16
std::vector< WeightMapping > HuggingFaceMapping(const HfModelConfig &config)
The manifest for a Llama-family Hugging Face checkpoint (model.embed_tokens.weight,...
std::function< Tensor(int64_t position, const Tensor &hidden)> HiddenStateHook
Reads or replaces a language model's residual stream during forward() (FEAT-3): called with a positio...
Definition causal_lm.hpp:37
WeightLoadReport LoadWeights(Module &model, const HfCheckpoint &checkpoint, const std::vector< WeightMapping > &mapping, const WeightLoadOptions &options={})
Loads checkpoint tensors into model's named parameters and buffers by mapping. Every tensor is read,...
OpType
The op-type tag a Node carries. Charter Part 2 §3: nodes are tagged by a small closed set of op types...
Definition op_type.hpp:19
std::unique_ptr< CausalLM > LoadCausalLM(const std::string &directory, DeviceBackend *backend)
Reads config.json and the weights in a downloaded Hugging Face model directory into a new CausalLM,...
std::function< std::vector< float >(const std::vector< int64_t > &tokens)> NextTokenLogitsFn
The model as generation sees it: the token sequence so far in, the logits for the next token out (one...
Definition generation.hpp:25
WeightTransform
How a source tensor becomes a target parameter.
Definition causal_lm.hpp:170
RMS normalization layer (Zhang & Sennrich, 2019) – this codebase's first normalization Module,...
A decoder-only language model's config.json, with the values Hugging Face leaves to the model class f...
Definition hf_model.hpp:33
Configuration for LRP relevance propagation: which rule a module applies and its hyperparameters....
Definition lrp_rule_config.hpp:72
Loading settings.
Definition causal_lm.hpp:191
bool strict
Every target must be loaded and every source used (or ignored); otherwise nothing is changed and the ...
Definition causal_lm.hpp:194
std::vector< std::string > ignore
Source tensors that may go unused, for example a duplicated lm_head.weight in a tied checkpoint.
Definition causal_lm.hpp:197
What a load did.
Definition causal_lm.hpp:201
std::vector< std::string > missing
Model parameters and buffers nothing loaded (empty when strict).
Definition causal_lm.hpp:204
std::vector< std::string > unused
Checkpoint tensors nothing used, ignored ones included.
Definition causal_lm.hpp:206
std::vector< std::string > loaded
Definition causal_lm.hpp:202
One entry of a name-mapping manifest.
Definition causal_lm.hpp:178
WeightTransform transform
Definition causal_lm.hpp:183
int64_t row_begin
Take only these rows of the source's first axis, before the transform: one part of a fused tensor suc...
Definition causal_lm.hpp:186
int64_t row_count
Definition causal_lm.hpp:187
std::string target
The model parameter's (or buffer's) name.
Definition causal_lm.hpp:182
std::string source
The checkpoint tensor's name.
Definition causal_lm.hpp:180
Language-model head that reuses an EmbeddingModule's table as its weight (LLM-2).