pulsatrix 1.1.0
Loading...
Searching...
No Matches
pulsatrix::NoiseSchedule Class Reference

The DDPM (Ho et al. 2020, arXiv:2006.11239) linear variance schedule plus the two tensor operations defined directly on top of it – closed-form forward noising and the reverse (sampling) step. More...

#include <noise_schedule.hpp>

Public Member Functions

 NoiseSchedule (int64_t num_timesteps, float beta_start=1e-4f, float beta_end=0.02f)
 Precomputes the linear beta schedule and its alpha / alpha_bar derivatives.
 
int64_t num_timesteps () const
 Number of diffusion steps T this schedule was built for.
 
float beta (int64_t t) const
 Variance beta_t at timestep t.
 
float alpha (int64_t t) const
 alpha_t = 1 - beta_t.
 
float alpha_bar (int64_t t) const
 alpha_bar_t = prod_{s=1..t} alpha_s (the cumulative product).
 
Tensor add_noise (const Tensor &x0, const Tensor &epsilon, int64_t t) const
 Closed-form forward (noising) sample: x_t = sqrt(alpha_bar_t) * x0 + sqrt(1 - alpha_bar_t) * epsilon.
 
Tensor denoise_step (const Tensor &x_t, const Tensor &predicted_epsilon, const Tensor &z, int64_t t) const
 One reverse (sampling) step, x_{t-1} = mean_term + sqrt(beta_t) * z where the mean term is (1/sqrt(alpha_t)) * (x_t - (beta_t/sqrt(1-alpha_bar_t)) * eps_theta).
 
ReverseStepCoefficients ddpm_coefficients (int64_t t) const
 denoise_step()'s coefficients: skip 1/sqrt(alpha_t), network beta_t / (sqrt(alpha_t) sqrt(1 - alpha_bar_t)), noise sqrt(beta_t).
 
ReverseStepCoefficients ddim_coefficients (int64_t t, int64_t s, float eta=0.0f) const
 A DDIM step (Song et al. 2021, arXiv:2010.02502, eq. 12) from t to an earlier s: x_s = sqrt(alpha_bar_s) x0_hat + sqrt(1 - alpha_bar_s - sigma^2) eps + sigma z with x0_hat = (x_t - sqrt(1 - alpha_bar_t) eps) / sqrt(alpha_bar_t) and sigma = eta sqrt((1 - alpha_bar_s) / (1 - alpha_bar_t)) sqrt(1 - alpha_bar_t / alpha_bar_s). s = 0 is the clean sample, alpha_bar_0 = 1, where sigma is zero.
 
Tensor ddim_step (const Tensor &x_t, const Tensor &predicted_epsilon, const Tensor &z, int64_t t, int64_t s, float eta=0.0f) const
 Applies ddim_coefficients(t, s, eta): skip * x_t - network * eps + noise * z.
 

Static Public Member Functions

static NoiseSchedule Cosine (int64_t num_timesteps, double offset=0.008, double max_beta=0.999)
 The cosine schedule of Nichol & Dhariwal 2021 (arXiv:2102.09672): alpha_bar_t = f(t) / f(0) with f(t) = cos^2(((t / T) + offset) / (1 + offset) * pi / 2), and beta_t = min(1 - alpha_bar_t / alpha_bar_{t-1}, max_beta). Computed in double.
 

Detailed Description

The DDPM (Ho et al. 2020, arXiv:2006.11239) linear variance schedule plus the two tensor operations defined directly on top of it – closed-form forward noising and the reverse (sampling) step.

beta_t is linearly spaced over t = 1 .. T from beta_start to beta_end (the paper's own choice of schedule), with alpha_t = 1 - beta_t and alpha_bar_t = prod_{s=1..t} alpha_s precomputed once at construction.

Note
1-indexed timesteps. t runs over [1, T], matching the published math verbatim, not the 0-based indexing of the underlying storage. Every accessor takes the published t and does the offset internally; nothing outside this class ever sees the 0-based index.
Not a Module, and has no parameters. Nothing here is learned and nothing here has a backward pass of its own – the schedule is a fixed, deterministic function of its three constructor arguments, precomputed at construction in the same spirit as RoPEModule's fixed non-learned rotation angles. The training objective a diffusion model is actually optimized against is a plain MSELoss between the true and predicted noise (L_simple), which needs no new loss class; the gradient path runs entirely through the caller's epsilon_theta network, never through this object.
No LRP rule of its own. A reverse step is skip * x_t - network * eps + noise * z (ReverseStepCoefficients), and trajectory LRP (trajectory_lrp.hpp, DIFF-5) splits relevance across those three terms. The schedule only supplies the coefficients.
**epsilon and z are caller-supplied, never sampled internally.** Identical convention to Reparameterize's epsilon: it keeps both operations deterministic and therefore directly testable against hand-computed values. Wiring real Gaussian sampling in is a demo's job, not this class's.

Constructor & Destructor Documentation

◆ NoiseSchedule()

pulsatrix::NoiseSchedule::NoiseSchedule ( int64_t  num_timesteps,
float  beta_start = 1e-4f,
float  beta_end = 0.02f 
)
explicit

Precomputes the linear beta schedule and its alpha / alpha_bar derivatives.

Parameters
num_timestepsNumber of diffusion steps T. Must be positive.
beta_startVariance at t = 1.
beta_endVariance at t = T. Must be strictly greater than beta_start.
Exceptions
std::invalid_argumentif num_timesteps <= 0 or beta_start >= beta_end – external boundary (constructor arguments can originate from the Python bindings with no upstream validation), same convention as every module constructor's argument checks.
Note
T == 1 is a degenerate but legal schedule: there is no interval to interpolate across, so beta_1 = beta_start (using the linear formula's own t = 1 endpoint rather than dividing by T - 1 == 0).

Member Function Documentation

◆ add_noise()

Tensor pulsatrix::NoiseSchedule::add_noise ( const Tensor &  x0,
const Tensor &  epsilon,
int64_t  t 
) const

Closed-form forward (noising) sample: x_t = sqrt(alpha_bar_t) * x0 + sqrt(1 - alpha_bar_t) * epsilon.

The closed form is why a diffusion model can be trained on a randomly chosen t without ever simulating the intermediate steps 1..t-1.

Parameters
x0Clean data tensor.
epsilonCaller-supplied noise, nominally ~ N(0, I). Must match x0's shape.
tTimestep in [1, T].
Returns
The noised tensor x_t, same shape as x0.
Exceptions
std::invalid_argumentif x0 and epsilon have different shapes – external boundary, same classification as MSELoss::forward's shape check.
std::out_of_rangeif t is outside [1, T].
Note
Device-generic: runs on Cpu, Cuda or Hip tensors (GPU-native-kernels Mission 1b); inputs must share one device.

◆ alpha()

float pulsatrix::NoiseSchedule::alpha ( int64_t  t) const

alpha_t = 1 - beta_t.

Parameters
tTimestep in [1, T].
Exceptions
std::out_of_rangeif t is outside [1, T].

◆ alpha_bar()

float pulsatrix::NoiseSchedule::alpha_bar ( int64_t  t) const

alpha_bar_t = prod_{s=1..t} alpha_s (the cumulative product).

Parameters
tTimestep in [1, T].
Exceptions
std::out_of_rangeif t is outside [1, T].

◆ beta()

float pulsatrix::NoiseSchedule::beta ( int64_t  t) const

Variance beta_t at timestep t.

Parameters
tTimestep in [1, T].
Exceptions
std::out_of_rangeif t is outside [1, T].

◆ Cosine()

static NoiseSchedule pulsatrix::NoiseSchedule::Cosine ( int64_t  num_timesteps,
double  offset = 0.008,
double  max_beta = 0.999 
)
static

The cosine schedule of Nichol & Dhariwal 2021 (arXiv:2102.09672): alpha_bar_t = f(t) / f(0) with f(t) = cos^2(((t / T) + offset) / (1 + offset) * pi / 2), and beta_t = min(1 - alpha_bar_t / alpha_bar_{t-1}, max_beta). Computed in double.

Exceptions
std::invalid_argumentif num_timesteps <= 0, offset < 0, or max_beta is outside (0, 1].
Note
alpha_bar() then follows the clipped betas, as in the paper's code: only the last steps are clipped, where f falls to zero.

◆ ddim_coefficients()

ReverseStepCoefficients pulsatrix::NoiseSchedule::ddim_coefficients ( int64_t  t,
int64_t  s,
float  eta = 0.0f 
) const

A DDIM step (Song et al. 2021, arXiv:2010.02502, eq. 12) from t to an earlier s: x_s = sqrt(alpha_bar_s) x0_hat + sqrt(1 - alpha_bar_s - sigma^2) eps + sigma z with x0_hat = (x_t - sqrt(1 - alpha_bar_t) eps) / sqrt(alpha_bar_t) and sigma = eta sqrt((1 - alpha_bar_s) / (1 - alpha_bar_t)) sqrt(1 - alpha_bar_t / alpha_bar_s). s = 0 is the clean sample, alpha_bar_0 = 1, where sigma is zero.

Parameters
eta0 is deterministic DDIM; 1 with s = t - 1 has the mean of a DDPM step and the posterior variance beta_tilde_t as its noise.
Exceptions
std::out_of_rangeif t is outside [1, T] or s outside [0, t).
std::invalid_argumentif eta < 0.

◆ ddim_step()

Tensor pulsatrix::NoiseSchedule::ddim_step ( const Tensor &  x_t,
const Tensor &  predicted_epsilon,
const Tensor &  z,
int64_t  t,
int64_t  s,
float  eta = 0.0f 
) const

Applies ddim_coefficients(t, s, eta): skip * x_t - network * eps + noise * z.

Exceptions
asddim_coefficients(), and std::invalid_argument if the shapes or devices differ.

◆ ddpm_coefficients()

ReverseStepCoefficients pulsatrix::NoiseSchedule::ddpm_coefficients ( int64_t  t) const

denoise_step()'s coefficients: skip 1/sqrt(alpha_t), network beta_t / (sqrt(alpha_t) sqrt(1 - alpha_bar_t)), noise sqrt(beta_t).

Exceptions
std::out_of_rangeif t is outside [1, T].

◆ denoise_step()

Tensor pulsatrix::NoiseSchedule::denoise_step ( const Tensor &  x_t,
const Tensor &  predicted_epsilon,
const Tensor &  z,
int64_t  t 
) const

One reverse (sampling) step, x_{t-1} = mean_term + sqrt(beta_t) * z where the mean term is (1/sqrt(alpha_t)) * (x_t - (beta_t/sqrt(1-alpha_bar_t)) * eps_theta).

Parameters
x_tCurrent noisy sample.
predicted_epsilonThe network's noise prediction at this timestep. Must match x_t's shape.
zCaller-supplied noise, nominally ~ N(0, I) for t > 1 and exactly zero at t == 1 (the final step is deterministic in the published algorithm). Must match x_t's shape. Zeroing z at t == 1 is the caller's responsibility and is deliberately not enforced here – consistent with every other sampled input in this mission being a caller-supplied deterministic value rather than something this class decides.
tTimestep in [1, T].
Returns
x_{t-1}, same shape as x_t.
Exceptions
std::invalid_argumentif the three shapes don't all match.
std::out_of_rangeif t is outside [1, T].
Note
Same raw-host-loop device guard rationale as add_noise().

◆ num_timesteps()

int64_t pulsatrix::NoiseSchedule::num_timesteps ( ) const
inline

Number of diffusion steps T this schedule was built for.


The documentation for this class was generated from the following file: