Adam optimizer – operates uniformly across any Module's parameters().
Adam (Kingma & Ba, 2015): per-parameter moving averages of gradient (m) and squared gradient (v),...
Definition adam_optimizer.hpp:20
Vendor-agnostic compute/memory backend. CPUBackend, CUDABackend (Phase 1.5), and HIPBackend (Phase 1....
Definition device_backend.hpp:219
y = x @ W + b, batched (x is (N, in_features), y is (N, out_features)) – migrated from the original u...
Definition linear_module.hpp:37
const Tensor & weight() const
Definition linear_module.hpp:91
MSE = mean((prediction - target)^2).
Definition mse_loss.hpp:19
Interface the training loop logs scalars/histograms through. Concrete writers (TensorBoard event form...
Definition metrics_sink.hpp:22
y = max(x, 0), elementwise. No parameters, no parameter gradients.
Definition relu_module.hpp:15
N-dimensional tensor. Owns its data buffer exclusively; a DeviceBackend* is injected (not owned) – th...
Definition tensor.hpp:29
A tiny MLP (Linear(2,4) -> ReLU -> Linear(4,1)) trained on XOR – the canonical not-linearly-separable...
Definition xor_training_example.hpp:32
XorNetwork(DeviceBackend *backend)
Constructs the network with fixed, non-zero initial weights.
LinearModule & linear2()
Definition xor_training_example.hpp:72
Tensor forward(const Tensor &input)
Runs the network forward.
const Tensor & linear1_weight() const
Test/inspection accessor.
Definition xor_training_example.hpp:61
LinearModule & linear1()
Mutable layer accessors – Population Based Training's own exploit step needs to read/write this netwo...
Definition xor_training_example.hpp:71
float train_step(const Tensor &input, const Tensor &target, AdamOptimizer &optimizer, MetricsSink &sink, int step)
Runs one training step: forward, loss, backward through every layer, one Adam update per layer's para...
Dense/fully-connected layer – the reference Module implementation.
Keeps monitoring/visualization tools out of the training core – same OCP/DIP pattern as DeviceBackend...
Definition acquisition_functions.hpp:16
ReLU activation – the second Module subclass, following LinearModule's pattern.