pulsatrix 1.1.0
Loading...
Searching...
No Matches
rope_module.hpp File Reference

Rotary Position Embedding – fixed per-position pair rotation, epsilon-rule LRP. More...

#include <optional>
#include <vector>
#include "pulsatrix/module.hpp"
Include dependency graph for rope_module.hpp:
This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Classes

class  pulsatrix::RoPEModule
 Rotary Position Embedding (RoPE, Su et al. 2021): a fixed, non-learnable, position-dependent rotation of each adjacent feature pair of a Q/K-shaped tensor. More...
 

Namespaces

namespace  pulsatrix
 

Enumerations

enum class  pulsatrix::RoPELayout { pulsatrix::AdjacentPairs , pulsatrix::RotateHalf }
 Which features RoPE rotates together (LLM-1). More...
 

Detailed Description

Rotary Position Embedding – fixed per-position pair rotation, epsilon-rule LRP.