Deep Learning Modules and Layers

Core tensor/graph infrastructure, layers, optimizers, losses, and device backends – the building blocks every other group is composed from. More...

Files
file	adam_optimizer.hpp
	Adam optimizer – operates uniformly across any Module's parameters().
file	assert.hpp
	PULSATRIX_ASSERT – debug-only invariant check for programmer errors, distinct from throw (used for caller-facing contract violations). See cpp_style_guide/context_style_project_conventions.md's assert-vs-throw table.
file	autograd.hpp
	Reverse-mode autodiff – walks a ComputationGraph backward, accumulating gradients via per-node backward functions supplied by the caller.
file	avg_pool2d_module.hpp
	2D average pooling, non-overlapping windows (stride == kernel), no padding.
file	batch_norm_module.hpp
	Batch normalization (Ioffe & Szegedy, 2015) – per-channel statistics computed across the batch and spatial dimensions jointly, unlike GroupNormModule's per-(batch-row, group) statistics.
file	bce_with_logits_loss.hpp
	Binary cross-entropy on raw logits – combined sigmoid + BCE, numerically stable.
file	calibration_loss.hpp
	Brier score – a proper-scoring-rule calibration loss (Brier, 1950).
file	computation_graph.hpp
	Owns and exposes graph structure – the interpretability substrate every explainer (Phase 2+) walks.
file	conv2d_module.hpp
	2D convolution – implemented via im2col + DeviceBackend::gemm (no new backend primitive).
file	cpu_backend.hpp
	CPU implementation of DeviceBackend – the first, reference DeviceBackend implementation.
file	cross_entropy_loss.hpp
	Softmax + negative log-likelihood classification loss.
file	cublas_check.hpp
	PULSATRIX_CUBLAS_CHECK – converts a cuBLAS call failure into a thrown C++ exception, parallel to PULSATRIX_CUDA_CHECK (cuda_check.hpp). Only compiled when PULSATRIX_ENABLE_CUDA is set.
file	cuda_backend.hpp
	CUDA implementation of DeviceBackend. Only compiled when PULSATRIX_ENABLE_CUDA is set.
file	cuda_check.hpp
	PULSATRIX_CUDA_CHECK – converts a CUDA runtime failure into a thrown C++ exception at the DeviceBackend boundary, per gpu_backend_programming/context_gpu_cuda_kernel_mechanics.md's Error Checking pattern. Only compiled when PULSATRIX_ENABLE_CUDA is set.
file	determinism.hpp
	One global seed for everything that isn't given its own, and a deterministic mode that forbids nondeterministic computation (roadmap FND-7).
file	device_backend.hpp
	Abstract interface isolating vendor-specific memory/compute operations from Tensor/ComputationGraph.
file	dropout_module.hpp
	Inverted dropout – the first module whose forward behavior genuinely differs between training and inference (Module::is_training()).
file	embedding_module.hpp
	Lookup-table (row-select) layer – y = W[index], batched (N, L) -> (N, L, embedding_dim).
file	flatten_module.hpp
	Reshape-only Module – flattens every non-batch dim of a (N, ...) input to (N, flattened_features), for chaining Conv2DModule's batched (N,C,H,W) output into a LinearModule's batched (N, in_features) input.
file	group_norm_module.hpp
	Group normalization (Wu & He, 2018) – rank-3 (C, H, W), matching Conv2DModule's convention, unlike RMSNormModule/LayerNormModule's rank-1 feature-vector scope.
file	gru_module.hpp
	Single-layer GRU – gated recurrence with a reset-gated candidate.
file	hip_backend.hpp
	HIP/ROCm implementation of DeviceBackend. Only compiled when PULSATRIX_ENABLE_HIP is set.
file	hip_check.hpp
	PULSATRIX_HIP_CHECK – converts a HIP runtime failure into a thrown C++ exception at the DeviceBackend boundary, per gpu_backend_programming/context_gpu_cuda_kernel_mechanics.md's Error Checking pattern. Only compiled when PULSATRIX_ENABLE_HIP is set.
file	hipblas_check.hpp
	PULSATRIX_HIPBLAS_CHECK – converts a hipBLAS call failure into a thrown C++ exception, parallel to PULSATRIX_HIP_CHECK (hip_check.hpp). Only compiled when PULSATRIX_ENABLE_HIP is set.
file	host_guard.hpp
	PULSATRIX_REQUIRE_HOST – always-on guard for code paths that dereference Tensor::data() on the host.
file	kl_divergence_loss.hpp
	VAE KL-divergence-to-standard-normal loss term.
file	layer_norm_module.hpp
	Layer normalization (Ba et al., 2016) – mean-centered/scaled RMSNormModule sibling.
file	linear_module.hpp
	Dense/fully-connected layer – the reference Module implementation.
file	lstm_module.hpp
	Single-layer LSTM – this codebase's first gated recurrent module.
file	mamba_module.hpp
	Mamba/S6 selective-state-space recurrence with the MambaLRP relevance rule.
file	max_pool2d_module.hpp
	2D max pooling, non-overlapping windows (stride == kernel), no padding.
file	metrics_sink.hpp
	Keeps monitoring/visualization tools out of the training core – same OCP/DIP pattern as DeviceBackend/ExplainerContext.
file	mnist_classifier_example.hpp
	Real MNIST classifier: Conv2DModule -> ReluModule -> FlattenModule -> LinearModule -> CrossEntropyLoss, mirroring XorNetwork's training-loop pattern and grad_cam_mnist_demo.cpp's network shape exactly.
file	mnist_loader.hpp
	Parses real MNIST IDX/ubyte files (fetched by tools/fetch_mnist.py) into Tensor images and integer labels.
file	module.hpp
	Abstract base every layer subclasses – NVI forward(), pure-virtual LRP contract.
file	mse_loss.hpp
	Mean squared error loss.
file	multihead_attention_module.hpp
	Multi-head scaled dot-product attention – this codebase's first Module composed out of other real Modules, plus AttnLRP's Eq. 15 bilinear relevance rule.
file	node.hpp
	Computation graph node – op type, shape, optional label, parent/child edges.
file	noise_schedule.hpp
	DDPM linear noise schedule: precomputed beta/alpha/alpha_bar, forward noising and reverse sampling steps.
file	op_type.hpp
	Closed set of operation categories every graph Node is tagged with.
file	relu_module.hpp
	ReLU activation – the second Module subclass, following LinearModule's pattern.
file	reparameterize.hpp
	VAE reparameterization trick: z = mu + exp(log_sigma) * epsilon.
file	residual_module.hpp
	Generic skip-connection wrapper – Phase 4's only mission, the ResNet-style residual block generalized past a fixed Conv-BN-ReLU stack to any Module.
file	retnet_module.hpp
	RetNet retention mechanism (recurrent mode), with an original derived LRP rule (no published rule exists for RetNet – see the class-level note).
file	rms_norm_module.hpp
	RMS normalization layer (Zhang & Sennrich, 2019) – this codebase's first normalization Module, establishing the Tier 1 pattern for GroupNorm/LayerNorm.
file	rnn_module.hpp
	Vanilla (Elman) recurrent layer – this codebase's first recurrent module.
file	rope_module.hpp
	Rotary Position Embedding – fixed per-position pair rotation, epsilon-rule LRP.
file	rwkv_module.hpp
	RWKV-4 time-mixing (WKV linear-attention) recurrence, with an original derived LRP rule adapting MambaLRP's detach-the-gate technique to the WKV quotient (see the class-level note).
file	safetensors.hpp
	Native safetensors reader and writer (roadmap IO-1): the one file format pulsatrix reads and writes for weights.
file	sequential_module.hpp
	Model container – chains a sequence of existing Modules.
file	sgd_optimizer.hpp
	Stochastic gradient descent – operates uniformly across any Module's parameters().
file	shape.hpp
	Tensor dimension arithmetic – rank, element count, per-dimension access.
file	sinusoidal_timestep_embedding.hpp
	Fixed sinusoidal encoding of a diffusion timestep, for conditioning a denoiser.
file	softmax_module.hpp
	Rank-agnostic softmax over the last axis, with AttnLRP's Eq. 13 DTD relevance rule.
file	swiglu_module.hpp
	SwiGLU gated feedforward block – second module composed from real LinearModule sub-objects, plus a diagonal specialization of AttnLRP's Eq. 15 bilinear rule.
file	tensor.hpp
	N-dimensional tensor – owns a buffer via DeviceBackend*, RAII (Rule of Five).
file	top_k.hpp
	Top-k selection along a tensor's last dimension, on any device (roadmap FND-3).
file	transformer_block.hpp
	Pre-LN transformer block – Phase 3's literal exit-gate deliverable, third and last composition mission (RMSNorm x2, MultiHeadAttentionModule, SwiGLUModule, plus the resolved residual-split LRP rule).
file	xor_training_example.hpp
	Phase 1's training-loop proof: Linear(2,4)->ReLU->Linear(4,1) learning XOR.

Detailed Description

Core tensor/graph infrastructure, layers, optimizers, losses, and device backends – the building blocks every other group is composed from.