47 [[nodiscard]]
bool feedforward()
const {
return feedforward_; }
52 [[nodiscard]]
bool merged()
const {
return merged_; }
53 [[nodiscard]]
bool enabled()
const {
return enabled_; }
78[[nodiscard]] std::vector<std::pair<std::string, LinearModule*>>
IA3Layers(
Module& model);
96void SaveIA3Adapter(
Module& model,
const std::string& directory,
const std::vector<WeightMapping>& mapping = {});
105void LoadIA3Adapter(
Module& model,
const std::string& directory,
const std::vector<WeightMapping>& mapping = {});
A Llama-family decoder-only language model built from a Hugging Face config, and loading its Hugging ...
Vendor-agnostic compute/memory backend. CPUBackend, CUDABackend (Phase 1.5), and HIPBackend (Phase 1....
Definition device_backend.hpp:248
One layer's rescaling vector l: (out) for an output-scaling layer, (in) for a feedforward one....
Definition ia3.hpp:42
bool enabled() const
Definition ia3.hpp:53
bool merged() const
Definition ia3.hpp:52
void set_scale(const std::vector< float > &values)
Replaces l.
const Tensor & scale_grad() const
Definition ia3.hpp:49
IA3Adapter(int64_t size, bool feedforward, DeviceBackend *backend, DeviceType device)
l = 1: the layer is unchanged.
const Tensor & scale() const
Definition ia3.hpp:48
bool feedforward() const
Definition ia3.hpp:47
y = x @ W + b, batched (x is (N, in_features), y is (N, out_features)) – migrated from the original u...
Definition linear_module.hpp:43
Base class for every layer type (LinearModule, Conv2DModule, activations, ...).
Definition module.hpp:78
N-dimensional tensor. Owns its data buffer exclusively; a DeviceBackend* is injected (not owned) – th...
Definition tensor.hpp:29
Abstract base every layer subclasses – NVI forward(), pure-virtual LRP contract.
Definition acquisition_functions.hpp:16
void RemoveIA3(Module &model)
Removes every vector; merged ones stay in the weights.
void SetIA3Enabled(Module &model, bool enabled)
Turns every vector on or off.
std::vector< std::pair< std::string, LinearModule * > > IA3Layers(Module &model)
The model's LinearModules with an IA³ vector, by name, in the model's order.
std::vector< std::string > ApplyIA3(Module &model, const IA3Config &config={})
Attaches an IA³ vector to each LinearModule config selects, then freezes everything else,...
void MergeIA3(Module &model)
Folds every vector into its layer's weights (and an output-scaling layer's bias).
DeviceType
Which physical device a Tensor's buffer resides on.
Definition device_backend.hpp:17
void LoadIA3Adapter(Module &model, const std::string &directory, const std::vector< WeightMapping > &mapping={})
Reads a PEFT IA³ adapter directory onto the model, telling feedforward layers by the vectors' shapes,...
void UnmergeIA3(Module &model)
Divides it back out, as PEFT does. Entries of l within 1e-8 of 0 can't be undone exactly.
void SaveIA3Adapter(Module &model, const std::string &directory, const std::vector< WeightMapping > &mapping={})
Writes the model's IA³ vectors as a PEFT adapter directory (adapter_model.safetensors with base_model...
Which layers IA³ rescales, PEFT style (see LoRAConfig::target_modules for the matching).
Definition ia3.hpp:28
std::vector< std::string > target_modules
The layers to rescale. Empty: the paper's choice, keys, values and the MLP's inner activations: every...
Definition ia3.hpp:32
std::vector< std::string > feedforward_modules
Those whose input is rescaled rather than their output. Each must be among the targets....
Definition ia3.hpp:35
N-dimensional tensor – owns a buffer via DeviceBackend*, RAII (Rule of Five).