rope_module.hpp File Reference

Rotary Position Embedding – fixed per-position pair rotation, epsilon-rule LRP. More...

#include <optional>
#include "pulsatrix/module.hpp"

Include dependency graph for rope_module.hpp:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Classes
class	pulsatrix::RoPEModule
	Rotary Position Embedding (RoPE, Su et al. 2021): a fixed, non-learnable, position-dependent rotation of each adjacent feature pair of a Q/K-shaped tensor. More...

Namespaces
namespace	pulsatrix

Detailed Description

Rotary Position Embedding – fixed per-position pair rotation, epsilon-rule LRP.