pulsatrix 1.1.0
Loading...
Searching...
No Matches
pulsatrix::FineTuneConfig Struct Reference

Training settings. The defaults are the recipe's. More...

#include <tagger_finetune_example.hpp>

Public Attributes

int64_t steps = 300
 
int64_t micro_batches = 2
 accumulated per optimizer step (TRN-5)
 
int64_t micro_batch_size = 8
 
float learning_rate = 1e-2f
 
int64_t warmup = 10
 then cosine decay to step steps (TRN-4)
 
float weight_decay = 0.01f
 AdamW, none on biases and norms (TRN-1, TRN-2)
 
float max_grad_norm = 1.0f
 (TRN-3)
 
uint64_t data_seed = 1000
 
int64_t stop_after = -1
 stop after this many steps and checkpoint to checkpoint_path
 
std::string checkpoint_path
 
std::string resume_from
 resume model, optimizer and schedule from this checkpoint (IO-2)
 

Detailed Description

Training settings. The defaults are the recipe's.

Member Data Documentation

◆ checkpoint_path

std::string pulsatrix::FineTuneConfig::checkpoint_path

◆ data_seed

uint64_t pulsatrix::FineTuneConfig::data_seed = 1000

◆ learning_rate

float pulsatrix::FineTuneConfig::learning_rate = 1e-2f

◆ max_grad_norm

float pulsatrix::FineTuneConfig::max_grad_norm = 1.0f

(TRN-3)

◆ micro_batch_size

int64_t pulsatrix::FineTuneConfig::micro_batch_size = 8

◆ micro_batches

int64_t pulsatrix::FineTuneConfig::micro_batches = 2

accumulated per optimizer step (TRN-5)

◆ resume_from

std::string pulsatrix::FineTuneConfig::resume_from

resume model, optimizer and schedule from this checkpoint (IO-2)

◆ steps

int64_t pulsatrix::FineTuneConfig::steps = 300

◆ stop_after

int64_t pulsatrix::FineTuneConfig::stop_after = -1

stop after this many steps and checkpoint to checkpoint_path

◆ warmup

int64_t pulsatrix::FineTuneConfig::warmup = 10

then cosine decay to step steps (TRN-4)

◆ weight_decay

float pulsatrix::FineTuneConfig::weight_decay = 0.01f

AdamW, none on biases and norms (TRN-1, TRN-2)


The documentation for this struct was generated from the following file: