pulsatrix 1.1.0
Loading...
Searching...
No Matches
pulsatrix::GenerationConfig Struct Reference

Decoding settings. The defaults decode greedily. Field names follow Hugging Face's GenerationConfig. More...

#include <generation.hpp>

Public Attributes

int64_t max_new_tokens = 32
 
int64_t min_new_tokens = 0
 EOS cannot be generated before this many new tokens.
 
std::vector< int64_t > eos_token_ids
 Generation stops after any of these is generated (it is kept in the output).
 
bool do_sample = false
 false: always the most likely token (greedy). true: sample.
 
float temperature = 1.0f
 Divides the logits; below 1 sharpens, above 1 flattens. Must be positive.
 
int64_t top_k = 0
 Keep only the top_k most likely tokens; 0 keeps all.
 
float top_p = 1.0f
 Keep the smallest set of most likely tokens whose probability reaches top_p; 1 keeps all.
 
float min_p = 0.0f
 Drop tokens less likely than min_p times the most likely one; 0 keeps all.
 
float repetition_penalty = 1.0f
 CTRL's penalty on every token already in the sequence, prompt included: positive logits are divided by it, negative ones multiplied; 1 is off.
 
std::optional< uint64_t > seed
 The sampling seed; unset draws one from the global stream (next_seed(), FND-7), so generation is reproducible after set_seed() either way.
 

Detailed Description

Decoding settings. The defaults decode greedily. Field names follow Hugging Face's GenerationConfig.

Member Data Documentation

◆ do_sample

bool pulsatrix::GenerationConfig::do_sample = false

false: always the most likely token (greedy). true: sample.

◆ eos_token_ids

std::vector<int64_t> pulsatrix::GenerationConfig::eos_token_ids

Generation stops after any of these is generated (it is kept in the output).

◆ max_new_tokens

int64_t pulsatrix::GenerationConfig::max_new_tokens = 32

◆ min_new_tokens

int64_t pulsatrix::GenerationConfig::min_new_tokens = 0

EOS cannot be generated before this many new tokens.

◆ min_p

float pulsatrix::GenerationConfig::min_p = 0.0f

Drop tokens less likely than min_p times the most likely one; 0 keeps all.

◆ repetition_penalty

float pulsatrix::GenerationConfig::repetition_penalty = 1.0f

CTRL's penalty on every token already in the sequence, prompt included: positive logits are divided by it, negative ones multiplied; 1 is off.

◆ seed

std::optional<uint64_t> pulsatrix::GenerationConfig::seed

The sampling seed; unset draws one from the global stream (next_seed(), FND-7), so generation is reproducible after set_seed() either way.

◆ temperature

float pulsatrix::GenerationConfig::temperature = 1.0f

Divides the logits; below 1 sharpens, above 1 flattens. Must be positive.

◆ top_k

int64_t pulsatrix::GenerationConfig::top_k = 0

Keep only the top_k most likely tokens; 0 keeps all.

◆ top_p

float pulsatrix::GenerationConfig::top_p = 1.0f

Keep the smallest set of most likely tokens whose probability reaches top_p; 1 keeps all.


The documentation for this struct was generated from the following file: