|
pulsatrix 1.1.0
|
LoRA's settings. The defaults are PEFT's LoraConfig's.
More...
#include <lora.hpp>
Public Attributes | |
| int64_t | rank = 8 |
| The rank r of the update. | |
| float | alpha = 8.0f |
| The update is scaled by alpha / rank. | |
| float | dropout = 0.0f |
| Dropout on the adapter's input, in training mode only. | |
| uint64_t | seed = 0 |
| Seeds A's initialization (and the dropout masks), the same on every platform. | |
| std::vector< std::string > | target_modules |
The layers ApplyLora() adapts, PEFT style: a layer matches a name if its own name is that name or ends with "." and that name (q_proj matches layers.0.mha.q_proj). Empty means every LinearModule but an output head named lm_head, as PEFT's "all-linear". (PEFT's own default for Llama is q_proj and v_proj only.) | |
LoRA's settings. The defaults are PEFT's LoraConfig's.
| float pulsatrix::LoRAConfig::alpha = 8.0f |
The update is scaled by alpha / rank.
| float pulsatrix::LoRAConfig::dropout = 0.0f |
Dropout on the adapter's input, in training mode only.
| int64_t pulsatrix::LoRAConfig::rank = 8 |
The rank r of the update.
| uint64_t pulsatrix::LoRAConfig::seed = 0 |
Seeds A's initialization (and the dropout masks), the same on every platform.
| std::vector<std::string> pulsatrix::LoRAConfig::target_modules |
The layers ApplyLora() adapts, PEFT style: a layer matches a name if its own name is that name or ends with "." and that name (q_proj matches layers.0.mha.q_proj). Empty means every LinearModule but an output head named lm_head, as PEFT's "all-linear". (PEFT's own default for Llama is q_proj and v_proj only.)