Single-layer LSTM – this codebase's first gated recurrent module. More...

#include <optional>
#include <initializer_list>
#include <vector>
#include "pulsatrix/module.hpp"

Include dependency graph for lstm_module.hpp:

Classes
class	pulsatrix::LSTMModule
	Standard 4-gate LSTM recurrence, h_0 = c_0 = 0 (zero-initialized, not learnable – the same deliberate scope cut RNNModule made, and the same thing that makes this module's conservation exact; see the LRP note below): i_t = sigmoid(x_t @ W_xi + h_{t-1} @ W_hi + b_i), f_t = sigmoid(x_t @ W_xf + h_{t-1} @ W_hf + b_f), g_t = tanh(x_t @ W_xg + h_{t-1} @ W_hg + b_g), o_t = sigmoid(x_t @ W_xo + h_{t-1} @ W_ho + b_o), c_t = f_t * c_{t-1} + i_t * g_t, h_t = o_t * tanh(c_t). Input (N, L, input_size) -> output (N, L, hidden_size), the full hidden-state sequence (matches RNNModule's convention). Single layer, no bidirectional/ multi-layer/variable-length/peephole support. More...

Detailed Description

Single-layer LSTM – this codebase's first gated recurrent module.

Classes