sinusoidal_timestep_embedding.hpp File Reference

Fixed sinusoidal encoding of a diffusion timestep, for conditioning a denoiser. More...

#include <cstdint>
#include "pulsatrix/device_backend.hpp"
#include "pulsatrix/tensor.hpp"

Include dependency graph for sinusoidal_timestep_embedding.hpp:

Go to the source code of this file.

Namespaces
namespace	pulsatrix

Functions
Tensor	pulsatrix::SinusoidalTimestepEmbedding (int64_t t, int64_t embedding_dim, DeviceBackend *backend, float base=10000.0f)
	Standard Transformer-style sinusoidal encoding of a diffusion timestep t: emb[2i] = sin(t / base^(2i/embedding_dim)), emb[2i+1] = cos(t / base^(2i/embedding_dim)) for i in [0, embedding_dim/2).

Detailed Description

Fixed sinusoidal encoding of a diffusion timestep, for conditioning a denoiser.