|
pulsatrix 1.1.0
|
Decoding settings. The defaults decode greedily. Field names follow Hugging Face's GenerationConfig. More...
#include <generation.hpp>
Public Attributes | |
| int64_t | max_new_tokens = 32 |
| int64_t | min_new_tokens = 0 |
| EOS cannot be generated before this many new tokens. | |
| std::vector< int64_t > | eos_token_ids |
| Generation stops after any of these is generated (it is kept in the output). | |
| bool | do_sample = false |
| false: always the most likely token (greedy). true: sample. | |
| float | temperature = 1.0f |
| Divides the logits; below 1 sharpens, above 1 flattens. Must be positive. | |
| int64_t | top_k = 0 |
| Keep only the top_k most likely tokens; 0 keeps all. | |
| float | top_p = 1.0f |
| Keep the smallest set of most likely tokens whose probability reaches top_p; 1 keeps all. | |
| float | min_p = 0.0f |
| Drop tokens less likely than min_p times the most likely one; 0 keeps all. | |
| float | repetition_penalty = 1.0f |
| CTRL's penalty on every token already in the sequence, prompt included: positive logits are divided by it, negative ones multiplied; 1 is off. | |
| std::optional< uint64_t > | seed |
| The sampling seed; unset draws one from the global stream (next_seed(), FND-7), so generation is reproducible after set_seed() either way. | |
Decoding settings. The defaults decode greedily. Field names follow Hugging Face's GenerationConfig.
| bool pulsatrix::GenerationConfig::do_sample = false |
false: always the most likely token (greedy). true: sample.
| std::vector<int64_t> pulsatrix::GenerationConfig::eos_token_ids |
Generation stops after any of these is generated (it is kept in the output).
| int64_t pulsatrix::GenerationConfig::max_new_tokens = 32 |
| int64_t pulsatrix::GenerationConfig::min_new_tokens = 0 |
EOS cannot be generated before this many new tokens.
| float pulsatrix::GenerationConfig::min_p = 0.0f |
Drop tokens less likely than min_p times the most likely one; 0 keeps all.
| float pulsatrix::GenerationConfig::repetition_penalty = 1.0f |
CTRL's penalty on every token already in the sequence, prompt included: positive logits are divided by it, negative ones multiplied; 1 is off.
| std::optional<uint64_t> pulsatrix::GenerationConfig::seed |
The sampling seed; unset draws one from the global stream (next_seed(), FND-7), so generation is reproducible after set_seed() either way.
| float pulsatrix::GenerationConfig::temperature = 1.0f |
Divides the logits; below 1 sharpens, above 1 flattens. Must be positive.
| int64_t pulsatrix::GenerationConfig::top_k = 0 |
Keep only the top_k most likely tokens; 0 keeps all.
| float pulsatrix::GenerationConfig::top_p = 1.0f |
Keep the smallest set of most likely tokens whose probability reaches top_p; 1 keeps all.