pulsatrix 1.1.0
Loading...
Searching...
No Matches
layer_student.hpp
Go to the documentation of this file.
1
32#pragma once
33
34#include <cstdint>
35#include <memory>
36#include <vector>
37
39#include "pulsatrix/tensor.hpp"
40
41namespace pulsatrix {
42
44[[nodiscard]] std::vector<int64_t> EveryOtherLayer(int64_t num_layers);
45
56[[nodiscard]] std::unique_ptr<CausalLM> MakeStudentFromLayers(CausalLM& teacher, const std::vector<int64_t>& layers,
57 DeviceBackend* backend);
58
66public:
67 explicit HiddenStateTap(DeviceBackend* backend);
68
69 [[nodiscard]] Tensor forward_at(const Tensor& hidden, int64_t start, int64_t prompt_length) override;
72 [[nodiscard]] Tensor backward(const Tensor& grad_output) override;
73 [[nodiscard]] Tensor propagate_relevance(const Tensor& relevance_out, const LRPRuleConfig& config) override;
74 [[nodiscard]] bool supports_lrp_rule(LRPRule) const override { return true; }
75 [[nodiscard]] OpType op_type() const override { return OpType::Elementwise; }
76 [[nodiscard]] std::optional<DeviceType> compute_device() const override { return backend_->device(); }
77 void release_activations() override;
78
81 [[nodiscard]] const Tensor& hidden() const;
84 void add_gradient(const Tensor& gradient);
85
86private:
87 DeviceBackend* backend_;
88 Tensor hidden_;
89 Tensor extra_;
90 bool has_hidden_ = false;
91 bool has_extra_ = false;
92};
93
101public:
109 [[nodiscard]] float forward(const Tensor& student, const Tensor& teacher, const std::vector<float>& mask = {});
112 [[nodiscard]] Tensor backward() const;
113
114private:
115 DeviceBackend* backend_;
116 Shape shape_ = Shape({0});
117 DeviceType device_ = DeviceType::Cpu;
118 std::vector<double> grad_;
119 bool has_forwarded_ = false;
120};
121
122} // namespace pulsatrix
A Llama-family decoder-only language model built from a Hugging Face config, and loading its Hugging ...
Token embedding, num_hidden_layers pre-norm transformer blocks, a final RMSNorm and a language-model ...
Definition causal_lm.hpp:82
Vendor-agnostic compute/memory backend. CPUBackend, CUDABackend (Phase 1.5), and HIPBackend (Phase 1....
Definition device_backend.hpp:248
virtual DeviceType device() const noexcept=0
Which device this backend's buffers reside on.
DistilBERT's cosine loss: the mean over kept rows of 1 โˆ’ cos(s, t), with s and t the student's and te...
Definition layer_student.hpp:100
HiddenStateCosineLoss(DeviceBackend *backend)
Tensor backward() const
Gradient with respect to the student's vectors, their shape.
float forward(const Tensor &student, const Tensor &teacher, const std::vector< float > &mask={})
An identity intervention that keeps the hidden states it sees and adds a gradient to the one flowing ...
Definition layer_student.hpp:65
const Tensor & hidden() const
The last forward()'s hidden states, (N, L, hidden_size).
OpType op_type() const override
This module's operation-category tag, for ComputationGraph node tagging.
Definition layer_student.hpp:75
Tensor backward(const Tensor &grad_output) override
grad_output plus the gradient given to add_gradient(), if any, which is then dropped.
HiddenStateTap(DeviceBackend *backend)
Tensor forward_at(const Tensor &hidden, int64_t start, int64_t prompt_length) override
The edit for hidden states covering positions [start, start + L) of a sequence whose prompt is prompt...
std::optional< DeviceType > compute_device() const override
The device this module computes on, so forward() can reject an input on another device before any ker...
Definition layer_student.hpp:76
void add_gradient(const Tensor &gradient)
A gradient for the next backward() to add, the hidden states' shape.
void release_activations() override
Frees the tensors forward() kept for backward() and the relevance passes; containers release their ch...
Tensor propagate_relevance(const Tensor &relevance_out, const LRPRuleConfig &config) override
Computes this module's contribution to LRP relevance propagation.
bool supports_lrp_rule(LRPRule) const override
Whether propagate_relevance() implements rule (no silent fallback: callers such as ExplainerContext::...
Definition layer_student.hpp:74
A trainable edit of the residual stream after a transformer block (TRN-9: LoReFT),...
Definition causal_lm.hpp:53
An N-dimensional shape. A plain aggregate of dimensions with no invariant beyond "non-negative dimens...
Definition shape.hpp:24
N-dimensional tensor. Owns its data buffer exclusively; a DeviceBackend* is injected (not owned) โ€“ th...
Definition tensor.hpp:29
Definition acquisition_functions.hpp:16
std::vector< int64_t > EveryOtherLayer(int64_t num_layers)
Layers 0, 2, 4, ... of a num_layers model: one out of two, DistilBERT's choice.
DeviceType
Which physical device a Tensor's buffer resides on.
Definition device_backend.hpp:17
OpType
The op-type tag a Node carries. Charter Part 2 ยง3: nodes are tagged by a small closed set of op types...
Definition op_type.hpp:19
LRPRule
The LRP rule family a module applies. Semantics follow Zennit 1.0.0 exactly (Anders et al....
Definition lrp_rule_config.hpp:21
std::unique_ptr< CausalLM > MakeStudentFromLayers(CausalLM &teacher, const std::vector< int64_t > &layers, DeviceBackend *backend)
A CausalLM with teacher's config and weights but only the blocks in layers, in that order: the studen...
Configuration for LRP relevance propagation: which rule a module applies and its hyperparameters....
Definition lrp_rule_config.hpp:72
N-dimensional tensor โ€“ owns a buffer via DeviceBackend*, RAII (Rule of Five).