151 [[nodiscard]]
const std::vector<std::string>&
names()
const {
return names_; }
152 [[nodiscard]]
bool contains(
const std::string& name)
const {
return shard_of_.count(name) != 0; }
154 [[nodiscard]]
const std::string&
shard_of(
const std::string& name)
const;
160 [[nodiscard]] std::vector<std::string>
shards()
const;
164 std::map<std::string, SafetensorsFile> files_;
165 std::map<std::string, std::string> shard_of_;
166 std::vector<std::string> names_;
167 std::vector<std::string> shard_order_;
Vendor-agnostic compute/memory backend. CPUBackend, CUDABackend (Phase 1.5), and HIPBackend (Phase 1....
Definition device_backend.hpp:248
A Hugging Face checkpoint directory's weights: model.safetensors, or the shards listed by model....
Definition hf_model.hpp:139
std::vector< std::string > shards() const
The shard files, in the order first listed.
const std::string & shard_of(const std::string &name) const
The shard file holding name.
const SafetensorsTensorInfo & info(const std::string &name) const
name's header entry (dtype, shape).
static HfCheckpoint Open(const std::string &directory)
Opens directory: the index if there is one, otherwise the single file.
Tensor tensor(const std::string &name, DeviceBackend *backend) const
Reads name into a Tensor; see SafetensorsFile::tensor for the dtypes it converts.
bool contains(const std::string &name) const
Definition hf_model.hpp:152
const std::vector< std::string > & names() const
Every tensor name, sorted.
Definition hf_model.hpp:151
N-dimensional tensor. Owns its data buffer exclusively; a DeviceBackend* is injected (not owned) – th...
Definition tensor.hpp:29
Multi-head scaled dot-product attention – this codebase's first Module composed out of other real Mod...
Definition acquisition_functions.hpp:16
AttentionConfig ToAttentionConfig(const HfModelConfig &config)
The attention layer the config describes: rotate-half RoPE (with its scaling), causal unless the conf...
HfModelConfig ParseHfConfig(std::string_view json)
Parses config.json text. A multimodal config's text_config is read for the language model.
std::vector< double > RopeInverseFrequencies(const HfModelConfig &config)
The RoPE frequency of each pair, head_dim / 2 of them, as Hugging Face computes them: rope_theta^(-2i...
HfModelConfig ReadHfConfig(const std::string &path)
ParseHfConfig on a file.
Native safetensors reader and writer (roadmap IO-1): the one file format pulsatrix reads and writes f...
Everything that shapes a MultiHeadAttentionModule (LLM-1).
Definition multihead_attention_module.hpp:31
A decoder-only language model's config.json, with the values Hugging Face leaves to the model class f...
Definition hf_model.hpp:33
float embed_scale
Token embeddings are multiplied by this: sqrt(hidden_size) for Gemma 3.
Definition hf_model.hpp:86
std::string model_type
For example "llama", "qwen2", "qwen3", "gemma3_text".
Definition hf_model.hpp:37
std::optional< int64_t > sliding_window
The attention window when sliding-window attention is actually used.
Definition hf_model.hpp:78
std::optional< int64_t > bos_token_id
Definition hf_model.hpp:72
bool causal
false: every position attends to every other. dLLM's A2D ("autoregressive to diffusion") check...
Definition hf_model.hpp:68
int64_t head_dim
head_dim, or hidden_size / num_attention_heads when absent.
Definition hf_model.hpp:45
std::vector< std::string > layer_types
layer_types, for example "sliding_attention" / "full_attention" per layer.
Definition hf_model.hpp:81
int64_t num_attention_heads
Definition hf_model.hpp:41
std::string dtype
torch_dtype (newer files: dtype), for example "bfloat16".
Definition hf_model.hpp:76
int64_t intermediate_size
Definition hf_model.hpp:39
int64_t hidden_size
Definition hf_model.hpp:38
bool mlp_bias
Definition hf_model.hpp:62
bool qk_norm
RMSNorm on each head's queries and keys (Qwen3, Gemma 3).
Definition hf_model.hpp:64
std::optional< HfRopeScaling > rope_scaling
The full-attention layers' RoPE scaling (all layers' when there is one kind).
Definition hf_model.hpp:52
std::vector< int64_t > eos_token_ids
eos_token_id, which may be one id or a list.
Definition hf_model.hpp:74
std::vector< std::string > unsupported
Everything in the config that pulsatrix can't run yet, each naming the roadmap item that adds it (for...
Definition hf_model.hpp:100
int64_t max_position_embeddings
Definition hf_model.hpp:71
bool is_sliding_layer(int64_t i) const
Whether layer i uses sliding-window attention: per layer_types when given, otherwise every layer when...
float rms_norm_eps
Definition hf_model.hpp:47
int64_t num_hidden_layers
Definition hf_model.hpp:40
bool out_bias
Output projection bias (Llama's attention_bias covers it; Qwen2 never has one).
Definition hf_model.hpp:61
float norm_weight_offset
RMSNorms scale by offset + w: 1 for Gemma 3, whose norm weights are stored centered on zero (LLM-9).
Definition hf_model.hpp:84
int64_t num_key_value_heads
num_key_value_heads, or num_attention_heads when absent.
Definition hf_model.hpp:43
std::optional< float > rope_local_base_freq
Sliding-window layers' own RoPE base, unscaled (Gemma 3's rope_local_base_freq, or rope_parameters....
Definition hf_model.hpp:55
std::string hidden_act
hidden_act (Gemma: hidden_activation), for example "silu".
Definition hf_model.hpp:70
std::string architecture
architectures[0], for example "LlamaForCausalLM".
Definition hf_model.hpp:35
bool tie_word_embeddings
Definition hf_model.hpp:56
bool qkv_bias
Q, K and V projection biases: attention_bias, or the architecture's own rule (Qwen2 always has them).
Definition hf_model.hpp:59
float rope_theta
rope_theta (or rope_parameters.rope_theta), 10000 when absent. With per-layer-type RoPE (Gemma 3) it ...
Definition hf_model.hpp:50
std::optional< float > query_pre_attn_scalar
Definition hf_model.hpp:79
bool post_norms
Gemma 3's sandwich norms: attention and MLP outputs are normalized again before each residual add.
Definition hf_model.hpp:89
int64_t vocab_size
Definition hf_model.hpp:46
A rope_scaling (or rope_parameters) entry other than the default.
Definition hf_model.hpp:20
float factor
Definition hf_model.hpp:23
std::string type
rope_type (older files: type), for example "llama3", "linear", "yarn".
Definition hf_model.hpp:22
float low_freq_factor
Definition hf_model.hpp:24
int64_t original_max_position_embeddings
Definition hf_model.hpp:26
float high_freq_factor
Definition hf_model.hpp:25
One tensor's header entry. Offsets are relative to the start of the data section.
Definition safetensors.hpp:38