pulsatrix 1.1.0
Loading...
Searching...
No Matches
pulsatrix::LoRAConfig Struct Reference

LoRA's settings. The defaults are PEFT's LoraConfig's. More...

#include <lora.hpp>

Public Attributes

int64_t rank = 8
 The rank r of the update.
 
float alpha = 8.0f
 The update is scaled by alpha / rank.
 
float dropout = 0.0f
 Dropout on the adapter's input, in training mode only.
 
uint64_t seed = 0
 Seeds A's initialization (and the dropout masks), the same on every platform.
 
std::vector< std::string > target_modules
 The layers ApplyLora() adapts, PEFT style: a layer matches a name if its own name is that name or ends with "." and that name (q_proj matches layers.0.mha.q_proj). Empty means every LinearModule but an output head named lm_head, as PEFT's "all-linear". (PEFT's own default for Llama is q_proj and v_proj only.)
 

Detailed Description

LoRA's settings. The defaults are PEFT's LoraConfig's.

Member Data Documentation

◆ alpha

float pulsatrix::LoRAConfig::alpha = 8.0f

The update is scaled by alpha / rank.

◆ dropout

float pulsatrix::LoRAConfig::dropout = 0.0f

Dropout on the adapter's input, in training mode only.

◆ rank

int64_t pulsatrix::LoRAConfig::rank = 8

The rank r of the update.

◆ seed

uint64_t pulsatrix::LoRAConfig::seed = 0

Seeds A's initialization (and the dropout masks), the same on every platform.

◆ target_modules

std::vector<std::string> pulsatrix::LoRAConfig::target_modules

The layers ApplyLora() adapts, PEFT style: a layer matches a name if its own name is that name or ends with "." and that name (q_proj matches layers.0.mha.q_proj). Empty means every LinearModule but an output head named lm_head, as PEFT's "all-linear". (PEFT's own default for Llama is q_proj and v_proj only.)


The documentation for this struct was generated from the following file: