A full fine-tuning recipe (roadmap TRN-6): pretrain a tiny transformer tagger on one rule, save it, reload it and fine-tune every parameter on a related rule.
More...
|
| void | pulsatrix::InitTagger (TinyTagger &tagger, uint64_t seed) |
| | Fills the tagger's parameters reproducibly on every platform: matrices uniform in +-1/sqrt(fan_in), normalization gains 1, biases 0.
|
| |
| TaggingBatch | pulsatrix::MakeTaggingBatch (TaggingRule rule, int64_t batch_size, uint64_t seed, DeviceBackend *backend=nullptr) |
| | A deterministic, platform-independent batch; lengths are 5 to kSeqLen.
|
| |
| std::vector< float > | pulsatrix::TrainTagger (TinyTagger &tagger, TaggingRule rule, const FineTuneConfig &config, DeviceBackend *backend) |
| | Trains every parameter of tagger on rule with the whole v1.1 training stack: AdamW with parameter groups, warmup plus cosine, gradient accumulation normalized by the window's token count, and gradient clipping.
|
| |
| float | pulsatrix::TaggingAccuracy (TinyTagger &tagger, TaggingRule rule, int64_t num_sequences) |
| | Fraction of real tokens tagged correctly on num_sequences fresh sequences.
|
| |
A full fine-tuning recipe (roadmap TRN-6): pretrain a tiny transformer tagger on one rule, save it, reload it and fine-tune every parameter on a related rule.
- Note
- Real pretrained models (SmolLM2, ResNet18) arrive with v1.2's import work (IO-3 to IO-5). Until then the "pretrained" model is one this recipe trains itself; the fine-tuning half is unchanged when a real checkpoint replaces it.
-
It is per-token tagging rather than next-token prediction because attention has no causal mask yet; with bidirectional attention a language-model target would be visible in the input.