Embedding lookup table, rank-2 input (N, L) of float-encoded indices -> rank-3 output (N, L, embedding_dim). Structurally unlike every other module in this codebase: forward is a pure selection (row copy), with no arithmetic mixing across input features.
More...
|
| | EmbeddingModule (int64_t num_embeddings, int64_t embedding_dim, DeviceBackend *backend) |
| | Constructs an embedding table with a zero-initialized weight matrix.
|
| |
| Tensor | backward (const Tensor &grad_output) override |
| | Scatter-adds grad_output into the corresponding rows of weight_grad_.
|
| |
| OpType | op_type () const override |
| | Embedding per charter's closed OpType set.
|
| |
| void | set_weight (std::initializer_list< float > values) |
| | Overwrites the weight buffer – test/initialization use only.
|
| |
| void | set_weight (const std::vector< float > &values) |
| | Vector overload for runtime-sized sources – see Tensor's own vector ctor.
|
| |
| const Tensor & | weight () const |
| |
| const Tensor & | weight_grad () const |
| |
| Tensor | propagate_relevance (const Tensor &relevance_out, const LRPRuleConfig &config) override |
| | Sum-over-embedding-dimension LRP relevance aggregation (Arras et al. 2017).
|
| |
| std::vector< NamedParamRef > | named_parameters () override |
| | This module's trainable parameters, each with its hierarchical name – the one place a module declares its parameters (roadmap FND-1).
|
| |
| void | release_activations () override |
| | Frees the tensors forward() kept for backward() and the relevance passes; containers release their children too. Inference that only needs the output calls it after each forward, so a deep model holds one layer's activations at a time (a 1024-token ESM-2 650M pass otherwise keeps over 10 GB of them).
|
| |
| virtual | ~Module ()=default |
| |
| Tensor | forward (const Tensor &input) |
| | Runs this module's forward computation.
|
| |
| virtual std::optional< DeviceType > | compute_device () const |
| | The device this module computes on, so forward() can reject an input on another device before any kernel sees it (roadmap FND-8).
|
| |
| std::pair< Tensor, NodeId > | forward_traced (const Tensor &input, NodeId input_node, ComputationGraph &graph, Autograd &autograd) |
| | Runs forward() while also registering a ComputationGraph node (tagged with this module's op_type(), parented to input_node) and wiring an Autograd backward function that reuses this module's own backward() – the opt-in traced/explainable path, per Phase 2 Mission 0.
|
| |
| virtual bool | supports_lrp_rule (LRPRule rule) const |
| | Whether propagate_relevance() implements rule (no silent fallback: callers such as ExplainerContext::relevance_pass() throw rather than run a module on a rule it does not implement).
|
| |
| virtual std::vector< NamedBufferRef > | named_buffers () |
| | This module's buffers: state a checkpoint must save that no optimizer updates.
|
| |
| virtual std::vector< NamedModuleRef > | named_children () |
| | This module's direct sub-modules, each under the prefix its parameters carry in named_parameters(), in the same order (TRN-7: how ApplyLora() finds a model's Linear layers). Default: none. NamedModules() walks the whole tree.
|
| |
| virtual std::vector< ParamRef > | parameters () |
| | This module's trainable parameters and their gradients, for an optimizer to update uniformly across module types.
|
| |
| void | set_requires_grad (bool requires_grad, const std::string &prefix="") |
| | Freezes (false) or unfreezes (true) parameters by name (roadmap FND-2).
|
| |
| virtual void | set_training (bool training) |
| | Sets this module's training/eval mode. Defaults to training (matches every mainstream framework's Module default).
|
| |
| bool | is_training () const |
| | Whether this module is currently in training mode.
|
| |
Embedding lookup table, rank-2 input (N, L) of float-encoded indices -> rank-3 output (N, L, embedding_dim). Structurally unlike every other module in this codebase: forward is a pure selection (row copy), with no arithmetic mixing across input features.
- Note
- Float-indices design decision: Tensor is float-only (no integer tensor type exists anywhere in this codebase). Each input element is resolved to an index via round-to-nearest (std::llround, not truncation – a caller passing 2.9999999f due to float round-trip from an integer source should not silently land on index 2), then bounds-checked against [0, num_embeddings). This is a genuinely new pattern in this codebase, not reused from any existing module.
-
propagate_relevance sums relevance over the embedding dimension per (n, l) position (Arras et al. 2017, "Explaining Recurrent Neural Network Predictions in
Sentiment Analysis" – already cited in this charter for RNN/LSTM): a token's total relevance is the sum of its embedding vector's per-dimension relevance values – there is no further "input" beneath a discrete token id to redistribute to. Conserves exactly by construction.
-
backward() uses scatter-add gradient accumulation (the first module in this codebase needing it – LinearModule/Conv2DModule's gradients are dense-matmul sums, not index-selected accumulation): multiple (n, l) positions referencing the same row each contribute additively into that row of weight_grad_. Gradient w.r.t. the input indices themselves is undefined (discrete, non-differentiable) – backward() returns an all-zero tensor matching the input shape, matching every mainstream framework's nn.Embedding behavior.