|
pulsatrix 1.1.0
|
Core tensor/graph infrastructure, layers, optimizers (SGD with momentum, Adam, AdamW, parameter groups, learning-rate schedules, gradient clipping), losses, checkpoints and safetensors I/O, seeding and deterministic mode, benchmarking, and device backends with their caching allocator – the building blocks every other group is composed from. More...
Files | |
| file | activation_module.hpp |
| A pointwise activation chosen at construction: ReLU, tanh, sigmoid, SiLU, exact GELU or GELU's tanh approximation (PLM-1). | |
| file | adam_optimizer.hpp |
| Adam optimizer – operates uniformly across any Module's parameters(). | |
| file | adaptive_avg_pool2d_module.hpp |
| Average pooling to a fixed output size, as PyTorch's AdaptiveAvgPool2d (KS-9). | |
| file | assert.hpp |
| PULSATRIX_ASSERT – debug-only invariant check for programmer errors, distinct from throw (used for caller-facing contract violations). See cpp_style_guide/context_style_project_conventions.md's assert-vs-throw table. | |
| file | attnlrp_parity.hpp |
| AttnLRP parity with LXT on a real language model (LLM-7): pulsatrix's relevance against LXT's own AttnLRP on the same Hugging Face model and tokens. | |
| file | autograd.hpp |
| Reverse-mode autodiff – walks a ComputationGraph backward, accumulating gradients via per-node backward functions supplied by the caller. | |
| file | avg_pool2d_module.hpp |
| 2D average pooling, non-overlapping windows (stride == kernel), no padding. | |
| file | batch_norm_module.hpp |
| Batch normalization (Ioffe & Szegedy, 2015) – per-channel statistics computed across the batch and spatial dimensions jointly, unlike GroupNormModule's per-(batch-row, group) statistics. | |
| file | bce_with_logits_loss.hpp |
| Binary cross-entropy on raw logits – combined sigmoid + BCE, numerically stable. | |
| file | benchmark.hpp |
Timing, benchmark reports (pulsatrix.benchmark.v1) and regression comparison. | |
| file | born_again.hpp |
| Born-again networks (Furlanello et al., ICML 2018; roadmap KD-4): a student the size of its teacher, trained on the teacher's outputs, then a teacher for the next generation. | |
| file | caching_allocator.hpp |
| A caching device-memory allocator (roadmap HIP-3): freed blocks are kept and reused for later requests of the same size class instead of going back to the driver. | |
| file | calibration_loss.hpp |
| Brier score – a proper-scoring-rule calibration loss (Brier, 1950). | |
| file | causal_lm.hpp |
| A Llama-family decoder-only language model built from a Hugging Face config, and loading its Hugging Face weights by name (IO-4). | |
| file | checkpoint.hpp |
| Saving and resuming models (roadmap IO-2): parameters and buffers in one safetensors file, optimizer state in a sibling file, and a format version with a migration table. | |
| file | computation_graph.hpp |
| Owns and exposes graph structure – the interpretability substrate every explainer (Phase 2+) walks. | |
| file | conv2d_module.hpp |
| 2D convolution – implemented via im2col + DeviceBackend::gemm (no new backend primitive). | |
| file | cpu_backend.hpp |
| CPU implementation of DeviceBackend – the first, reference DeviceBackend implementation. | |
| file | cross_entropy_loss.hpp |
| Softmax + negative log-likelihood classification loss. | |
| file | cublas_check.hpp |
| PULSATRIX_CUBLAS_CHECK – converts a cuBLAS call failure into a thrown C++ exception, parallel to PULSATRIX_CUDA_CHECK (cuda_check.hpp). Only compiled when PULSATRIX_ENABLE_CUDA is set. | |
| file | cuda_backend.hpp |
| CUDA implementation of DeviceBackend. Only compiled when PULSATRIX_ENABLE_CUDA is set. | |
| file | cuda_check.hpp |
| PULSATRIX_CUDA_CHECK – converts a CUDA runtime failure into a thrown C++ exception at the DeviceBackend boundary, per gpu_backend_programming/context_gpu_cuda_kernel_mechanics.md's Error Checking pattern. Only compiled when PULSATRIX_ENABLE_CUDA is set. | |
| file | determinism.hpp |
| One global seed for everything that isn't given its own, and a deterministic mode that forbids nondeterministic computation (roadmap FND-7). | |
| file | device_backend.hpp |
| Abstract interface isolating vendor-specific memory/compute operations from Tensor/ComputationGraph. | |
| file | distillation_loss.hpp |
| Knowledge distillation (roadmap KD-1): a student learns from a teacher's softened outputs (Hinton, Vinyals and Dean, 2015), optionally mixed with the hard labels. | |
| file | dropout_module.hpp |
| Inverted dropout – the first module whose forward behavior genuinely differs between training and inference (Module::is_training()). | |
| file | embedding_module.hpp |
| Lookup-table (row-select) layer – y = W[index], batched (N, L) -> (N, L, embedding_dim). | |
| file | encoder_block.hpp |
| A configurable transformer encoder layer (PLM-1): LayerNorm or RMSNorm, before or after each sublayer, and a plain or gated MLP. ESM-2's layer, BERT's layer and the Llama-style pre-norm layer are all settings of it. | |
| file | encoder_lm.hpp |
Masked-language-model encoders: ESM-2 protein language models (PLM-2), Hugging Face EsmForMaskedLM checkpoints loaded, run, trained and explained natively, and MDLM masked diffusion language models (DIFF-2). | |
| file | feature_distillation.hpp |
| Distilling a teacher's intermediate features, not only its outputs (roadmap KD-3): FitNets' hint loss (Romero et al., ICLR 2015) and attention transfer (Zagoruyko and Komodakis, ICLR 2017). | |
| file | feed_forward_module.hpp |
The plain transformer MLP, Linear -> activation -> Linear, of BERT, ESM and the original Transformer (PLM-1). The gated variant is SwiGLUModule. | |
| file | flatten_module.hpp |
| Reshape-only Module – flattens every non-batch dim of a (N, ...) input to (N, flattened_features), for chaining Conv2DModule's batched (N,C,H,W) output into a LinearModule's batched (N, in_features) input. | |
| file | generation.hpp |
| Text generation from a language model: greedy decoding and seeded sampling with temperature, top-k, top-p and min-p, repetition penalty, and EOS handling (LLM-4). | |
| file | golden_logits.hpp |
| The golden-logit harness: a loaded model's logits against Hugging Face's on real text (LLM-6). | |
| file | grad_clipping.hpp |
| Global gradient-norm clipping (roadmap TRN-3). | |
| file | group_norm_module.hpp |
| Group normalization (Wu & He, 2018) – rank-3 (C, H, W), matching Conv2DModule's convention, unlike RMSNormModule/LayerNormModule's rank-1 feature-vector scope. | |
| file | gru_module.hpp |
| Single-layer GRU – gated recurrence with a reset-gated candidate. | |
| file | hf_model.hpp |
Hugging Face checkpoints: config.json and single or sharded safetensors (IO-5). | |
| file | hip_backend.hpp |
| HIP/ROCm implementation of DeviceBackend. Only compiled when PULSATRIX_ENABLE_HIP is set. | |
| file | hip_check.hpp |
| PULSATRIX_HIP_CHECK – converts a HIP runtime failure into a thrown C++ exception at the DeviceBackend boundary, per gpu_backend_programming/context_gpu_cuda_kernel_mechanics.md's Error Checking pattern. Only compiled when PULSATRIX_ENABLE_HIP is set. | |
| file | hipblas_check.hpp |
| PULSATRIX_HIPBLAS_CHECK – converts a hipBLAS call failure into a thrown C++ exception, parallel to PULSATRIX_HIP_CHECK (hip_check.hpp). Only compiled when PULSATRIX_ENABLE_HIP is set. | |
| file | host_guard.hpp |
| PULSATRIX_REQUIRE_HOST – always-on guard for code paths that dereference Tensor::data() on the host. | |
| file | ia3.hpp |
| IA³ (Liu et al., 2022): fine-tune a frozen model by learning one rescaling vector per adapted Linear layer, on its output (keys, values) or its input (the MLP's down projection), and save and load it as a PEFT adapter. | |
| file | kl_divergence_loss.hpp |
| VAE KL-divergence-to-standard-normal loss term. | |
| file | kv_cache.hpp |
| Preallocated key/value cache for incremental attention (LLM-5). | |
| file | layer_norm_module.hpp |
| Layer normalization (Ba et al., 2016) – mean-centered/scaled RMSNormModule sibling. | |
| file | layer_student.hpp |
| DistilBERT's recipe for a smaller language model (Sanh et al., 2019; roadmap KD-4): a student with some of the teacher's layers, started from those layers' weights, and a cosine loss that aligns the two models' hidden states. | |
| file | linear_module.hpp |
| Dense/fully-connected layer – the reference Module implementation. | |
| file | lm_scoring.hpp |
| Prompt/completion data for a causal language model (LLM-10): read it from JSONL, batch it with the prompt masked out of the loss, and score it, as log P(completion | prompt) or as the probabilities of a few single-token choices ("P(True)"). | |
| file | lora.hpp |
| LoRA (Hu et al., 2021): fine-tune a frozen model through a low-rank update on each of its Linear layers, merge it into the weights or take it out again, and save and load it as a PEFT adapter. | |
| file | loreft.hpp |
| LoReFT (Wu et al., 2024): fine-tune a frozen language model by editing its hidden states in a learned low-rank subspace, at a few positions after chosen layers, and explain the edit with LRP. | |
| file | lr_scheduler.hpp |
| Learning-rate schedules (warmup, constant, linear, cosine) and a scheduler that applies one to an optimizer and its parameter groups (roadmap TRN-4). | |
| file | lstm_module.hpp |
| Single-layer LSTM – this codebase's first gated recurrent module. | |
| file | mamba_module.hpp |
| Mamba/S6 selective-state-space recurrence with the MambaLRP relevance rule. | |
| file | masked_diffusion.hpp |
| Sampling from a masked diffusion language model (DIFF-2): start from a sequence with masked positions and fill them over a number of steps, recording the whole trajectory so it can be explained afterwards (DIFF-3). DPLM's remasking sampler (DIFF-4) also takes positions back. | |
| file | max_pool2d_module.hpp |
| 2D max pooling with a stride and padding (non-overlapping windows by default). | |
| file | metrics_sink.hpp |
| Keeps monitoring/visualization tools out of the training core – same OCP/DIP pattern as DeviceBackend/ExplainerContext. | |
| file | mnist_classifier_example.hpp |
| Real MNIST classifier: Conv2DModule -> ReluModule -> FlattenModule -> LinearModule -> CrossEntropyLoss, mirroring XorNetwork's training-loop pattern and grad_cam_mnist_demo.cpp's network shape exactly. | |
| file | mnist_loader.hpp |
| Parses real MNIST IDX/ubyte files (fetched by tools/fetch_mnist.py) into Tensor images and integer labels. | |
| file | module.hpp |
| Abstract base every layer subclasses – NVI forward(), pure-virtual LRP contract. | |
| file | mse_loss.hpp |
| Mean squared error loss. | |
| file | multihead_attention_module.hpp |
| Multi-head scaled dot-product attention – this codebase's first Module composed out of other real Modules, plus AttnLRP's Eq. 15 bilinear relevance rule. | |
| file | node.hpp |
| Computation graph node – op type, shape, optional label, parent/child edges. | |
| file | noise_schedule.hpp |
| DDPM noise schedules (linear and cosine): precomputed beta/alpha/alpha_bar, forward noising, and DDPM and DDIM reverse steps. | |
| file | op_type.hpp |
| Closed set of operation categories every graph Node is tagged with. | |
| file | param_groups.hpp |
| Optimizer parameter groups: a learning rate and weight decay per set of parameters, chosen by name (roadmap TRN-1). | |
| file | protein_training.hpp |
| Training protein language models (PLM-7): ESM-2's masked-LM data pipeline (masking, cropping, cluster-weighted sampling), its initialization, a training step and evaluation, and heads for fine-tuning on per-protein and per-residue labels. | |
| file | relu_module.hpp |
| ReLU activation – the second Module subclass, following LinearModule's pattern. | |
| file | reparameterize.hpp |
| VAE reparameterization trick: z = mu + exp(log_sigma) * epsilon. | |
| file | residual_module.hpp |
| Generic skip-connection wrapper – Phase 4's only mission, the ResNet-style residual block generalized past a fixed Conv-BN-ReLU stack to any Module. | |
| file | retnet_module.hpp |
| RetNet retention mechanism (recurrent mode), with an original derived LRP rule (no published rule exists for RetNet – see the class-level note). | |
| file | rms_norm_module.hpp |
| RMS normalization layer (Zhang & Sennrich, 2019) – this codebase's first normalization Module, establishing the Tier 1 pattern for GroupNorm/LayerNorm. | |
| file | rnn_module.hpp |
| Vanilla (Elman) recurrent layer – this codebase's first recurrent module. | |
| file | rope_module.hpp |
| Rotary Position Embedding – fixed per-position pair rotation, epsilon-rule LRP. | |
| file | rwkv_module.hpp |
| RWKV-4 time-mixing (WKV linear-attention) recurrence, with an original derived LRP rule adapting MambaLRP's detach-the-gate technique to the WKV quotient (see the class-level note). | |
| file | safetensors.hpp |
| Native safetensors reader and writer (roadmap IO-1): the one file format pulsatrix reads and writes for weights. | |
| file | sequential_module.hpp |
| Model container – chains a sequence of existing Modules. | |
| file | sgd_optimizer.hpp |
| Stochastic gradient descent – operates uniformly across any Module's parameters(). | |
| file | shape.hpp |
| Tensor dimension arithmetic – rank, element count, per-dimension access. | |
| file | sinusoidal_timestep_embedding.hpp |
| Fixed sinusoidal encoding of a diffusion timestep, for conditioning a denoiser. | |
| file | soft_decision_tree.hpp |
| A soft decision tree (Frosst and Hinton, "Distilling a Neural Network Into a Soft
Decision Tree", 2017): an interpretable student for a network teacher (roadmap KD-3). | |
| file | softmax_module.hpp |
| Rank-agnostic softmax over the last axis, with AttnLRP's Eq. 13 DTD relevance rule. | |
| file | swiglu_module.hpp |
SwiGLU gated feedforward block – second module composed from real LinearModule sub-objects, plus a diagonal specialization of AttnLRP's Eq. 15 bilinear rule. | |
| file | tagger_finetune_example.hpp |
| A full fine-tuning recipe (roadmap TRN-6): pretrain a tiny transformer tagger on one rule, save it, reload it and fine-tune every parameter on a related rule. | |
| file | tensor.hpp |
| N-dimensional tensor – owns a buffer via DeviceBackend*, RAII (Rule of Five). | |
| file | tied_lm_head_module.hpp |
| Language-model head that reuses an EmbeddingModule's table as its weight (LLM-2). | |
| file | token_cross_entropy_loss.hpp |
| Cross-entropy over a batch of tokens, with padding ignored and a caller-chosen normalizer for exact gradient accumulation (roadmap TRN-5). | |
| file | top_k.hpp |
| Top-k selection along a tensor's last dimension, on any device (roadmap FND-3). | |
| file | transformer_block.hpp |
| Pre-LN transformer block – Phase 3's literal exit-gate deliverable, third and last composition mission (RMSNorm x2, MultiHeadAttentionModule, SwiGLUModule, plus the resolved residual-split LRP rule). | |
| file | vision_models.hpp |
| torchvision's ResNet and VGG, built from pulsatrix layers with torchvision's parameter names, so their published ImageNet weights load once converted to safetensors (KS-9). | |
| file | xor_training_example.hpp |
| Phase 1's training-loop proof: Linear(2,4)->ReLU->Linear(4,1) learning XOR. | |
Core tensor/graph infrastructure, layers, optimizers (SGD with momentum, Adam, AdamW, parameter groups, learning-rate schedules, gradient clipping), losses, checkpoints and safetensors I/O, seeding and deterministic mode, benchmarking, and device backends with their caching allocator – the building blocks every other group is composed from.