pulsatrix 1.1.0
Loading...
Searching...
No Matches
kv_cache.hpp File Reference

Preallocated key/value cache for incremental attention (LLM-5). More...

#include <cstdint>
#include "pulsatrix/tensor.hpp"
Include dependency graph for kv_cache.hpp:
This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Classes

class  pulsatrix::KVCache
 One attention layer's keys and values for the positions processed so far, in buffers allocated once for max_length positions, so generation never reallocates or copies the past. More...
 

Namespaces

namespace  pulsatrix
 

Detailed Description

Preallocated key/value cache for incremental attention (LLM-5).