Masked-language-model encoders: ESM-2 protein language models (PLM-2), Hugging Face EsmForMaskedLM checkpoints loaded, run, trained and explained natively, and MDLM masked diffusion language models (DIFF-2).
More...
|
| EncoderLMConfig | pulsatrix::ParseEsmConfig (std::string_view json) |
| | Reads an ESM config.json.
|
| |
| EncoderLMConfig | pulsatrix::ReadEsmConfig (const std::string &path) |
| | ParseEsmConfig on a file.
|
| |
| EncoderLMConfig | pulsatrix::ParseMdlmConfig (std::string_view json) |
| | Reads an MDLM config.json (kuleshov-group/mdlm-owt and models trained with the same code): a bidirectional transformer with rotate-half RoPE, LayerNorms without biases, bias-free attention, a tanh-GELU MLP four times the width, and an untied linear output layer. The mask token is the vocabulary's last id.
|
| |
| std::vector< WeightMapping > | pulsatrix::EsmMapping (const EncoderLMConfig &config) |
| | The name mapping from an EsmForMaskedLM checkpoint to EncoderLM. Q/K/V, output, MLP and LM-head dense weights are transposed.
|
| |
| std::vector< std::string > | pulsatrix::EsmIgnoredTensors (const EncoderLMConfig &config) |
| | Checkpoint tensors an ESM-2 EncoderLM doesn't use: the absolute-position table and ids (unused with rotary), each layer's rotary inv_freq (read separately), and the contact head (LoadEsmContactHead reads it).
|
| |
| std::vector< WeightMapping > | pulsatrix::MdlmMapping (const EncoderLMConfig &config) |
| | The name mapping from an MDLM checkpoint to EncoderLM. The fused attn_qkv is split into q, k and v; linear weights are transposed. The LayerNorm biases and the adaLN modulation have no counterpart here: LoadEncoderLM folds them in (see there).
|
| |
| std::unique_ptr< EncoderLM > | pulsatrix::LoadEncoderLM (const std::string &directory, DeviceBackend *backend) |
| | Loads a masked LM directory: config.json and safetensors weights (one file or sharded). The config's model_type picks the model.
|
| |
Masked-language-model encoders: ESM-2 protein language models (PLM-2), Hugging Face EsmForMaskedLM checkpoints loaded, run, trained and explained natively, and MDLM masked diffusion language models (DIFF-2).