pulsatrix 1.1.0
Loading...
Searching...
No Matches
pulsatrix::TranscoderOptions Struct Reference

#include <transcoder.hpp>

Public Attributes

int64_t k = 32
 Latents kept per input (a TopK activation, as Paulo et al. use).
 
bool skip = false
 A skip transcoder: add x W_skip, a linear map from input to output that starts at zero, so the latents only need to explain what isn't linear.
 
int64_t k_aux = 0
 AuxK, as TopKSparseAutoencoder: the k_aux largest dead latents predict the error (0: half the output dimension), weighted by aux_coefficient (0 turns it off).
 
float aux_coefficient = 1.0f / 32.0f
 
int64_t dead_after = 10'000'000
 
uint64_t seed = 0
 

Member Data Documentation

◆ aux_coefficient

float pulsatrix::TranscoderOptions::aux_coefficient = 1.0f / 32.0f

◆ dead_after

int64_t pulsatrix::TranscoderOptions::dead_after = 10'000'000

◆ k

int64_t pulsatrix::TranscoderOptions::k = 32

Latents kept per input (a TopK activation, as Paulo et al. use).

◆ k_aux

int64_t pulsatrix::TranscoderOptions::k_aux = 0

AuxK, as TopKSparseAutoencoder: the k_aux largest dead latents predict the error (0: half the output dimension), weighted by aux_coefficient (0 turns it off).

◆ seed

uint64_t pulsatrix::TranscoderOptions::seed = 0

◆ skip

bool pulsatrix::TranscoderOptions::skip = false

A skip transcoder: add x W_skip, a linear map from input to output that starts at zero, so the latents only need to explain what isn't linear.


The documentation for this struct was generated from the following file: