retnet_module.hpp File Reference

RetNet retention mechanism (recurrent mode), with an original derived LRP rule (no published rule exists for RetNet – see the class-level note). More...

#include <optional>
#include <initializer_list>
#include <vector>
#include "pulsatrix/module.hpp"

Include dependency graph for retnet_module.hpp:

Go to the source code of this file.

Classes
class	pulsatrix::RetNetModule
	Core RetNet retention block (Sun et al. 2023, arXiv:2307.08621), recurrent mode, input (N, L, d_model) -> output (N, L, d_model). More...

Namespaces
namespace	pulsatrix

Detailed Description

RetNet retention mechanism (recurrent mode), with an original derived LRP rule (no published rule exists for RetNet – see the class-level note).