pulsatrix 1.1.0
Loading...
Searching...
No Matches
masked_diffusion.hpp File Reference

Sampling from a masked diffusion language model (DIFF-2): start from a sequence with masked positions and fill them over a number of steps, recording the whole trajectory so it can be explained afterwards (DIFF-3). DPLM's remasking sampler (DIFF-4) also takes positions back. More...

#include <cstdint>
#include <functional>
#include <optional>
#include <vector>
#include "pulsatrix/causal_lm.hpp"
#include "pulsatrix/encoder_lm.hpp"
Include dependency graph for masked_diffusion.hpp:
This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Classes

struct  pulsatrix::MaskedDiffusionConfig
 Sampling settings. The defaults fill greedily by confidence. More...
 
struct  pulsatrix::MaskedDiffusionWrite
 One position filled at one step. More...
 
struct  pulsatrix::MaskedDiffusionStep
 One step that filled at least one position (under Remasking, any step). More...
 
struct  pulsatrix::MaskedDiffusionResult
 A sampled sequence and how it was produced. More...
 

Namespaces

namespace  pulsatrix
 

Typedefs

using pulsatrix::DenoiserFn = std::function< std::vector< float >(const std::vector< int64_t > &tokens)>
 The model as the sampler sees it: the current sequence (masked positions included) in, logits for every position out, L * vocab values, row-major. MakeDenoiser builds one for an EncoderLM or a bidirectional CausalLM.
 

Enumerations

enum class  pulsatrix::UnmaskingSchedule { pulsatrix::Confidence , pulsatrix::Ancestral , pulsatrix::Remasking }
 How the sampler chooses which masked positions to fill at each step. More...
 

Functions

MaskedDiffusionResult pulsatrix::SampleMaskedDiffusion (const DenoiserFn &model, const std::vector< int64_t > &tokens, const MaskedDiffusionConfig &config)
 Fills every masked position of tokens over the configured steps.
 
DenoiserFn pulsatrix::MakeDenoiser (EncoderLM &model, DeviceBackend *backend)
 A DenoiserFn for an EncoderLM: one forward pass over the sequence, (1, L) to logits (1, L, vocab). The model isn't owned and must outlive the function.
 
DenoiserFn pulsatrix::MakeDenoiser (CausalLM &model, DeviceBackend *backend)
 A DenoiserFn for a CausalLM loaded with bidirectional attention, such as an A2D diffusion model (HfModelConfig::causal false). The model must outlive the function.
 

Detailed Description

Sampling from a masked diffusion language model (DIFF-2): start from a sequence with masked positions and fill them over a number of steps, recording the whole trajectory so it can be explained afterwards (DIFF-3). DPLM's remasking sampler (DIFF-4) also takes positions back.