Customization: Extending pulsatrix¶
You extend pulsatrix by subclassing. To add a layer, a metrics sink or a device backend, you
subclass Module, MetricsSink or DeviceBackend and override their virtual methods. There is
no runtime plugin or factory registry: nothing like Register("my_layer", ...) and no
string-keyed lookups. Why no runtime registry explains the choice.
Adding a new layer¶
Subclass Module (module.hpp)
and implement four pure-virtual methods:
#include "pulsatrix/module.hpp"
class MyActivation : public Module {
public:
explicit MyActivation(DeviceBackend* backend) : backend_(backend) {}
[[nodiscard]] Tensor backward(const Tensor& grad_output) override {
// gradient w.r.t. this module's input, from state cached in forward_impl()
}
[[nodiscard]] OpType op_type() const override { return OpType::Activation; }
// Lets Module::forward() check that the input is on this layer's device.
[[nodiscard]] std::optional<DeviceType> compute_device() const override { return backend_->device(); }
[[nodiscard]] Tensor propagate_relevance(const Tensor& relevance_out,
const LRPRuleConfig& config) override {
// LRP: split relevance_out among this module's inputs (see below)
}
protected:
[[nodiscard]] Tensor forward_impl(const Tensor& input) override {
// the forward computation; Module::forward() calls this after checking the input
}
private:
DeviceBackend* backend_;
};
compute_device() is optional, but without it Module::forward() can't check that the input
lives on the same device as the layer, and a CPU tensor reaching a GPU layer fails much later
and less clearly. A layer with parameters usually returns its weight's device instead.
The LRP rule is required¶
propagate_relevance() is pure virtual, so a layer without an LRP rule doesn't compile. This
is deliberate: pulsatrix never falls back to a generic rule that may not fit the layer.
- Use a rule from the literature and cite it in the header comment.
- If the right rule for your layer isn't known yet, throw from
propagate_relevance()with a message that says so. Don't return a plausible-looking approximation.
Supporting more than the epsilon rule¶
LRPRuleConfig tells your layer which rule to apply (config.rule) and that rule's parameters.
By default a Module supports only LRPRule::Epsilon. If your layer implements other rules,
override supports_lrp_rule():
[[nodiscard]] bool supports_lrp_rule(LRPRule rule) const override {
return rule == LRPRule::Epsilon || rule == LRPRule::Gamma;
}
LRP::explain() checks every layer before it starts propagating, and throws if one doesn't
support the rule it was asked for. A layer that only passes relevance through unchanged, like
ReluModule or FlattenModule, can return true for every rule. The
LRP guide describes the existing rules.
If your layer has its own variants that aren't LRP rules, add an enum constructor parameter
rather than a new LRPRuleConfig field. ConjunctionModule::TNorm
(Product/Lukasiewicz/Godel, see Neuro-Symbolic Reasoning)
is an example.
Parameters¶
If your layer has trainable parameters, override named_parameters() (the default returns none)
so optimizers can update them and checkpoints, freezing and parameter groups can find them by
name. Use PyTorch's names where your layer has a PyTorch counterpart (weight, bias), and in a
layer that holds other modules, add each child's parameters under its name with
append_named_parameters:
std::vector<NamedParamRef> named_parameters() override {
std::vector<NamedParamRef> params{{"scale", {&scale_, &scale_grad_}}};
append_named_parameters(params, "proj", proj_); // proj.weight, proj.bias
return params;
}
parameters() returns the same tensors in the same order without the names. Layers written
before named_parameters() existed may still override parameters() instead; they keep
training, but report no names of their own. Inside a container they get positional names
(2.0, 2.1, ...), which are stable only as long as the parameter order is.
Buffers and eval mode¶
State that isn't trained but must be saved, such as BatchNorm's running statistics, is a
buffer. Override named_buffers() so checkpoints store it; a layer that holds other modules
adds theirs with append_named_buffers(out, "child", child_). Without it, a checkpoint
silently drops that state. If your layer holds other modules, also override set_training()
to pass the mode on to each of them, as SequentialModule does, so set_training(false)
reaches every BatchNorm and Dropout inside.
Adding a new metrics sink¶
Training loops report metrics such as the loss through a MetricsSink. To send them somewhere
new, implement its two methods:
#include "pulsatrix/metrics_sink.hpp"
class MyMetricsSink : public MetricsSink {
public:
void log_scalar(const std::string& tag, double value, int step) override { /* ... */ }
void log_histogram(const std::string& tag, const Tensor& values, int step) override { /* ... */ }
};
Pass your sink to train_step() (for example XorNetwork::train_step() or MnistConvNet's).
Two sinks ship with the library: NoOpMetricsSink, which discards everything, and
ImPlotMetricsSink, which buffers series for live plots (see
Visualization). An ImPlotMetricsSink's log can be saved as a
pulsatrix.training_log.v1 document (ToTrainingLogDocument) and replayed into any sink
later (ReplayTrainingLog).
Adding a new device backend¶
Implement DeviceBackend, the interface that CPUBackend and the CUDA and HIP backends
implement. Tensor and ComputationGraph depend only on this interface, never on a specific
backend.
Every backend must behave the same way:
- Results. The same results as the CPU up to floating-point rounding (GPU reductions add in a different order), and the same results from run to run: no atomics in reductions.
- Errors. Throw on failure.
allocate()returns nullptr only for a 0-byte request. - Ordering. Ops may run asynchronously, but a
copy()to the host,dot()andsum()must return finished results, andfree()must be safe on memory that queued work still uses. The HIP backend shows one way: a single in-order stream, waits only where the host reads, andPULSATRIX_HIP_SYNC_DEBUG=1to wait after every op while hunting a fault (GPU Profiling). - Memory.
CachingAllocator(caching_allocator.hpp) is device-independent: hand it your raw allocate, free and synchronize functions to reuse freed blocks (GPU Profiling). - Determinism. When
deterministic()is on, use only paths whose results don't vary from run to run; the HIP and CUDA backends turn off BLAS atomics, for example. - Selection.
top_k_rows()ranks NaN above every number and keeps the lower index first on ties, so every backend picks the same entries in the same order.
This section is about writing a new backend. To build the existing CUDA or HIP backends, see Getting Started.
Why no runtime registry¶
A string-keyed registry would let you add layers without recompiling. The cost is that the
compiler could no longer check that every Module has an LRP rule. That check is the core
promise of the library, so every extension point uses ordinary subclassing and virtual
dispatch instead.