|
pulsatrix 1.1.0
|
Prompt/completion data for a causal language model (LLM-10): read it from JSONL, batch it with the prompt masked out of the loss, and score it, as log P(completion | prompt) or as the probabilities of a few single-token choices ("P(True)"). More...
#include <cstdint>#include <string>#include <vector>#include "pulsatrix/causal_lm.hpp"#include "pulsatrix/json.hpp"#include "pulsatrix/tensor.hpp"#include "pulsatrix/text_tokenizer.hpp"
Go to the source code of this file.
Classes | |
| struct | pulsatrix::PromptCompletion |
| One example: the prompt's token ids, then the completion's. More... | |
| struct | pulsatrix::CausalLmBatch |
| A right-padded training batch. More... | |
| struct | pulsatrix::ChoiceScores |
| Scores of single-token choices after each prompt. Indexed [example][choice]. More... | |
Namespaces | |
| namespace | pulsatrix |
Functions | |
| std::vector< JsonValue > | pulsatrix::ReadJsonl (const std::string &path) |
| Reads a JSONL file: one JSON value a line. Blank lines are skipped. | |
| std::vector< PromptCompletion > | pulsatrix::TokenizePromptCompletions (const std::vector< JsonValue > &rows, const TextTokenizer &tokenizer, const std::string &prompt_key="prompt", const std::string &completion_key="completion", bool add_special_tokens=false) |
| Tokenizes each row's prompt and completion strings (objects with those keys). The two are tokenized separately, so a completion like " Yes" keeps its own tokens whatever the prompt ends with. | |
| CausalLmBatch | pulsatrix::CollatePromptCompletions (const std::vector< PromptCompletion > &examples, DeviceBackend *backend, int64_t pad_id=0) |
| Builds a batch for TokenCrossEntropyLoss: position i predicts token i + 1, and only the completion's tokens count. The completion's last token is never an input (nothing follows it), so each row is one token shorter than prompt + completion. | |
| std::vector< double > | pulsatrix::CompletionLogProbs (CausalLM &model, const std::vector< PromptCompletion > &examples, DeviceBackend *backend, int64_t batch_size=8) |
| log P(completion | prompt) under the model, summed over the completion's tokens, in double. Runs forward passes only, batch_size examples at a time, grouped by length. | |
| ChoiceScores | pulsatrix::ScoreChoices (CausalLM &model, const std::vector< std::vector< int64_t > > &prompts, const std::vector< int64_t > &choice_tokens, DeviceBackend *backend, int64_t batch_size=8) |
| Scores the next token after each prompt against a fixed set of single-token choices, such as " Yes" and " No". For two choices, choice_log_probs[n][0] - choice_log_probs[n][1] is the log-odds of the first. | |
| int64_t | pulsatrix::SingleTokenId (const TextTokenizer &tokenizer, const std::string &text) |
| The id of a string that must tokenize to exactly one token, such as " Yes". | |
Prompt/completion data for a causal language model (LLM-10): read it from JSONL, batch it with the prompt masked out of the loss, and score it, as log P(completion | prompt) or as the probabilities of a few single-token choices ("P(True)").
Fine-tuning a model to answer a question trains only the answer: CollatePromptCompletions() sets every target outside the completion to TokenCrossEntropyLoss::kIgnoreIndex, so the loss is the completion's log loss and nothing else. Scoring reads the same positions back.
Batches are right-padded. Causal attention never lets a real token attend to the padding after it, so padding changes no real position's logits (tests/lm_scoring_test.cpp checks a padded batch against each example alone).
The scorers copy only the logit rows they need to the host, gathered on the device: a batch's full logits are (N, L, vocab), about 400 MB for SmolLM2 at N = 8, L = 256.