Training settings. The defaults are the recipe's.
More...
#include <tagger_finetune_example.hpp>
Training settings. The defaults are the recipe's.
◆ checkpoint_path
| std::string pulsatrix::FineTuneConfig::checkpoint_path |
◆ data_seed
| uint64_t pulsatrix::FineTuneConfig::data_seed = 1000 |
◆ learning_rate
| float pulsatrix::FineTuneConfig::learning_rate = 1e-2f |
◆ max_grad_norm
| float pulsatrix::FineTuneConfig::max_grad_norm = 1.0f |
◆ micro_batch_size
| int64_t pulsatrix::FineTuneConfig::micro_batch_size = 8 |
◆ micro_batches
| int64_t pulsatrix::FineTuneConfig::micro_batches = 2 |
accumulated per optimizer step (TRN-5)
◆ resume_from
| std::string pulsatrix::FineTuneConfig::resume_from |
resume model, optimizer and schedule from this checkpoint (IO-2)
◆ steps
| int64_t pulsatrix::FineTuneConfig::steps = 300 |
◆ stop_after
| int64_t pulsatrix::FineTuneConfig::stop_after = -1 |
stop after this many steps and checkpoint to checkpoint_path
◆ warmup
| int64_t pulsatrix::FineTuneConfig::warmup = 10 |
then cosine decay to step steps (TRN-4)
◆ weight_decay
| float pulsatrix::FineTuneConfig::weight_decay = 0.01f |
AdamW, none on biases and norms (TRN-1, TRN-2)
The documentation for this struct was generated from the following file: