|
pulsatrix 1.1.0
|
Preallocated key/value cache for incremental attention (LLM-5). More...


Go to the source code of this file.
Classes | |
| class | pulsatrix::KVCache |
One attention layer's keys and values for the positions processed so far, in buffers allocated once for max_length positions, so generation never reallocates or copies the past. More... | |
Namespaces | |
| namespace | pulsatrix |
Preallocated key/value cache for incremental attention (LLM-5).