|
pulsatrix 1.1.0
|
Hugging Face checkpoints: config.json and single or sharded safetensors (IO-5).
More...
#include <cstdint>#include <map>#include <memory>#include <optional>#include <string>#include <vector>#include "pulsatrix/multihead_attention_module.hpp"#include "pulsatrix/safetensors.hpp"

Go to the source code of this file.
Classes | |
| struct | pulsatrix::HfRopeScaling |
A rope_scaling (or rope_parameters) entry other than the default. More... | |
| struct | pulsatrix::HfModelConfig |
A decoder-only language model's config.json, with the values Hugging Face leaves to the model class filled in: every field is what the model actually uses. More... | |
| class | pulsatrix::HfCheckpoint |
A Hugging Face checkpoint directory's weights: model.safetensors, or the shards listed by model.safetensors.index.json (IO-5). Shards are memory-mapped, so opening a checkpoint reads only headers, and a tensor's bytes are paged in when it is read. More... | |
Namespaces | |
| namespace | pulsatrix |
Functions | |
| HfModelConfig | pulsatrix::ParseHfConfig (std::string_view json) |
Parses config.json text. A multimodal config's text_config is read for the language model. | |
| HfModelConfig | pulsatrix::ReadHfConfig (const std::string &path) |
| ParseHfConfig on a file. | |
| std::vector< double > | pulsatrix::RopeInverseFrequencies (const HfModelConfig &config) |
The RoPE frequency of each pair, head_dim / 2 of them, as Hugging Face computes them: rope_theta^(-2i / head_dim), then rescaled by rope_scaling when it is "linear" (all divided by the factor) or "llama3" (low frequencies divided by the factor, high ones kept, the band between interpolated; LLM-6). | |
| AttentionConfig | pulsatrix::ToAttentionConfig (const HfModelConfig &config) |
| The attention layer the config describes: rotate-half RoPE (with its scaling), causal unless the config says otherwise, its head counts, biases, QK-Norm and epsilon, and Gemma's score scale and QK-Norm offset. For a model whose layers differ, this is a full-attention layer. | |
| AttentionConfig | pulsatrix::ToAttentionConfig (const HfModelConfig &config, int64_t layer) |
Layer layer's attention (LLM-9): a sliding-window layer gets its window and its own RoPE base, unscaled; a full-attention layer is ToAttentionConfig(config). | |
Hugging Face checkpoints: config.json and single or sharded safetensors (IO-5).