pulsatrix 1.1.0
Loading...
Searching...
No Matches
pulsatrix::SoftDecisionTree Class Reference

Frosst and Hinton's soft decision tree, (N, inputs) to (N, classes) log-probabilities. More...

#include <soft_decision_tree.hpp>

Inheritance diagram for pulsatrix::SoftDecisionTree:
Collaboration diagram for pulsatrix::SoftDecisionTree:

Public Member Functions

 SoftDecisionTree (int64_t inputs, int64_t classes, DeviceBackend *backend, SoftDecisionTreeOptions options={})
 
Tensor backward (const Tensor &grad_output) override
 Computes the gradient w.r.t. this module's input, given the gradient w.r.t. its output. Must be called after forward() – uses state cached from that call.
 
Tensor propagate_relevance (const Tensor &relevance_out, const LRPRuleConfig &config) override
 Computes this module's contribution to LRP relevance propagation.
 
bool supports_lrp_rule (LRPRule) const override
 Whether propagate_relevance() implements rule (no silent fallback: callers such as ExplainerContext::relevance_pass() throw rather than run a module on a rule it does not implement).
 
OpType op_type () const override
 This module's operation-category tag, for ComputationGraph node tagging.
 
std::vector< NamedParamRef > named_parameters () override
 gates.weight, gates.bias and leaves.
 
std::vector< NamedModuleRef > named_children () override
 This module's direct sub-modules, each under the prefix its parameters carry in named_parameters(), in the same order (TRN-7: how ApplyLora() finds a model's Linear layers). Default: none. NamedModules() walks the whole tree.
 
int depth () const
 
int64_t num_inner () const
 
int64_t num_leaves () const
 
int64_t num_inputs () const
 
int64_t num_classes () const
 
const SoftDecisionTreeOptions & options () const
 
LinearModule & gates ()
 The gates: column i of its (inputs, inner) weight and entry i of its bias are node i's w_i and b_i.
 
Tensor & leaf_logits ()
 The leaves' logits φ, (leaves, classes).
 
std::vector< double > LeafDistributions () const
 The leaves' distributions softmax(φ), row-major (leaves, classes).
 
const std::vector< double > & right_probabilities () const
 From the last forward(): each input's probability of going right at each inner node, (N, inner), and of reaching each leaf, (N, leaves), row-major.
 
const std::vector< double > & leaf_probabilities () const
 
double penalty () const
 The last forward()'s balance penalty C, λ included; 0 when λ is 0. Add it to the loss you report: backward() adds its gradient.
 
std::vector< int64_t > MostProbableLeaf () const
 For each input of the last forward(), the leaf its most probable path ends at: the greedy path, taking the likelier branch at every node, as Frosst and Hinton do.
 
std::vector< std::pair< int64_t, bool > > PathTo (int64_t leaf) const
 The inner nodes on the path from the root to leaf, root first, and whether the path goes right at each.
 
- Public Member Functions inherited from pulsatrix::Module
virtual ~Module ()=default
 
Tensor forward (const Tensor &input)
 Runs this module's forward computation.
 
virtual std::optional< DeviceType > compute_device () const
 The device this module computes on, so forward() can reject an input on another device before any kernel sees it (roadmap FND-8).
 
std::pair< Tensor, NodeId > forward_traced (const Tensor &input, NodeId input_node, ComputationGraph &graph, Autograd &autograd)
 Runs forward() while also registering a ComputationGraph node (tagged with this module's op_type(), parented to input_node) and wiring an Autograd backward function that reuses this module's own backward() – the opt-in traced/explainable path, per Phase 2 Mission 0.
 
virtual std::vector< NamedBufferRef > named_buffers ()
 This module's buffers: state a checkpoint must save that no optimizer updates.
 
virtual std::vector< ParamRef > parameters ()
 This module's trainable parameters and their gradients, for an optimizer to update uniformly across module types.
 
void set_requires_grad (bool requires_grad, const std::string &prefix="")
 Freezes (false) or unfreezes (true) parameters by name (roadmap FND-2).
 
virtual void set_training (bool training)
 Sets this module's training/eval mode. Defaults to training (matches every mainstream framework's Module default).
 
bool is_training () const
 Whether this module is currently in training mode.
 
virtual void release_activations ()
 Frees the tensors forward() kept for backward() and the relevance passes; containers release their children too. Inference that only needs the output calls it after each forward, so a deep model holds one layer's activations at a time (a 1024-token ESM-2 650M pass otherwise keeps over 10 GB of them).
 

Protected Member Functions

Tensor forward_impl (const Tensor &input) override
 The actual forward computation. Called by forward() after precondition checks.
 

Detailed Description

Frosst and Hinton's soft decision tree, (N, inputs) to (N, classes) log-probabilities.

Note
The routing and the leaves are computed on the host in double precision; the gates' linear layer runs on the backend's device, and the output comes back on the input's device.
LRP isn't defined for it: propagate_relevance() throws. Explain it with its paths, or with gradient explainers.

Constructor & Destructor Documentation

◆ SoftDecisionTree()

pulsatrix::SoftDecisionTree::SoftDecisionTree ( int64_t  inputs,
int64_t  classes,
DeviceBackend *  backend,
SoftDecisionTreeOptions  options = {} 
)

Member Function Documentation

◆ backward()

Tensor pulsatrix::SoftDecisionTree::backward ( const Tensor &  grad_output)
overridevirtual

Computes the gradient w.r.t. this module's input, given the gradient w.r.t. its output. Must be called after forward() – uses state cached from that call.

Parameters
grad_outputGradient w.r.t. this module's output.
Returns
Gradient w.r.t. this module's input.
Note
Promoted to the base class in Phase 2 Mission 0 – every existing subclass (LinearModule/ReluModule/Conv2DModule) already implemented this exact signature independently; making it virtual lets graph-wiring code (Module::forward_traced) call it polymorphically through a Module* without knowing the concrete subclass, the same way propagate_relevance already works.

Implements pulsatrix::Module.

◆ depth()

int pulsatrix::SoftDecisionTree::depth ( ) const
inline

◆ forward_impl()

Tensor pulsatrix::SoftDecisionTree::forward_impl ( const Tensor &  input)
overrideprotectedvirtual

The actual forward computation. Called by forward() after precondition checks.

Implements pulsatrix::Module.

◆ gates()

LinearModule & pulsatrix::SoftDecisionTree::gates ( )
inline

The gates: column i of its (inputs, inner) weight and entry i of its bias are node i's w_i and b_i.

◆ leaf_logits()

Tensor & pulsatrix::SoftDecisionTree::leaf_logits ( )
inline

The leaves' logits φ, (leaves, classes).

◆ leaf_probabilities()

const std::vector< double > & pulsatrix::SoftDecisionTree::leaf_probabilities ( ) const
inline

◆ LeafDistributions()

std::vector< double > pulsatrix::SoftDecisionTree::LeafDistributions ( ) const

The leaves' distributions softmax(φ), row-major (leaves, classes).

◆ MostProbableLeaf()

std::vector< int64_t > pulsatrix::SoftDecisionTree::MostProbableLeaf ( ) const

For each input of the last forward(), the leaf its most probable path ends at: the greedy path, taking the likelier branch at every node, as Frosst and Hinton do.

◆ named_children()

std::vector< NamedModuleRef > pulsatrix::SoftDecisionTree::named_children ( )
overridevirtual

This module's direct sub-modules, each under the prefix its parameters carry in named_parameters(), in the same order (TRN-7: how ApplyLora() finds a model's Linear layers). Default: none. NamedModules() walks the whole tree.

Note
The containers that build language models and their layers report their children: SequentialModule, ResidualModule, the attention, MLP and transformer blocks, CausalLM and EncoderLM. A module that doesn't is a leaf to NamedModules().

Reimplemented from pulsatrix::Module.

◆ named_parameters()

std::vector< NamedParamRef > pulsatrix::SoftDecisionTree::named_parameters ( )
overridevirtual

gates.weight, gates.bias and leaves.

Reimplemented from pulsatrix::Module.

◆ num_classes()

int64_t pulsatrix::SoftDecisionTree::num_classes ( ) const
inline

◆ num_inner()

int64_t pulsatrix::SoftDecisionTree::num_inner ( ) const
inline

◆ num_inputs()

int64_t pulsatrix::SoftDecisionTree::num_inputs ( ) const
inline

◆ num_leaves()

int64_t pulsatrix::SoftDecisionTree::num_leaves ( ) const
inline

◆ op_type()

OpType pulsatrix::SoftDecisionTree::op_type ( ) const
inlineoverridevirtual

This module's operation-category tag, for ComputationGraph node tagging.

Returns
This module's OpType (see op_type.hpp's closed set).
Note
Added in Phase 2 Mission 0 alongside backward() – lets graph-wiring code tag nodes generically through a Module* rather than switching on concrete subclass.

Implements pulsatrix::Module.

◆ options()

const SoftDecisionTreeOptions & pulsatrix::SoftDecisionTree::options ( ) const
inline

◆ PathTo()

std::vector< std::pair< int64_t, bool > > pulsatrix::SoftDecisionTree::PathTo ( int64_t  leaf) const

The inner nodes on the path from the root to leaf, root first, and whether the path goes right at each.

◆ penalty()

double pulsatrix::SoftDecisionTree::penalty ( ) const
inline

The last forward()'s balance penalty C, λ included; 0 when λ is 0. Add it to the loss you report: backward() adds its gradient.

◆ propagate_relevance()

Tensor pulsatrix::SoftDecisionTree::propagate_relevance ( const Tensor &  relevance_out,
const LRPRuleConfig &  config 
)
overridevirtual

Computes this module's contribution to LRP relevance propagation.

Parameters
relevance_outRelevance at this module's output.
configSelects the LRP rule variant.
Returns
Relevance at this module's input.

Implements pulsatrix::Module.

◆ right_probabilities()

const std::vector< double > & pulsatrix::SoftDecisionTree::right_probabilities ( ) const
inline

From the last forward(): each input's probability of going right at each inner node, (N, inner), and of reaching each leaf, (N, leaves), row-major.

◆ supports_lrp_rule()

bool pulsatrix::SoftDecisionTree::supports_lrp_rule ( LRPRule  rule) const
inlineoverridevirtual

Whether propagate_relevance() implements rule (no silent fallback: callers such as ExplainerContext::relevance_pass() throw rather than run a module on a rule it does not implement).

Note
Default: only LRPRule::Epsilon, the rule every module implements. Overridden by LinearModule / Conv2DModule (all rules), by parameter-free pass-through modules whose relevance rule does not read the config at all, and by SequentialModule (all layers).

Reimplemented from pulsatrix::Module.


The documentation for this class was generated from the following file: