text_collate.hpp File Reference

PadCollate – right-pads variable-length token sequences into one batch Tensor. More...

#include "pulsatrix/collate.hpp"

Include dependency graph for text_collate.hpp:

Go to the source code of this file.

Namespaces
namespace	pulsatrix

Functions
CollateFn	pulsatrix::PadCollate (float pad_index=0.0f)
	Builds a CollateFn that right-pads variable-length token sequences (sample.fields[0], shape (1, seq_len) – TextDataset::get()'s output) to the batch's own max length, producing one (N, max_len) Tensor, plus a (N,) length field recording each sample's real (pre-padding) length. This is the CollateFn extension point Phase 1's architecture design reserved for ragged/variable-length modalities (PyTorch's collate_fn equivalent) – exercised here for the first time, with zero changes needed to Dataset/DataLoader/Batch themselves.

Detailed Description

PadCollate – right-pads variable-length token sequences into one batch Tensor.