|
pulsatrix 1.1.0
|
The JumpReLU sparse autoencoder (FEAT-4; Rajamanoharan et al., "Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders", arXiv 2407.14435): a learned threshold per latent, trained against an L0 penalty through straight-through estimators. More...
#include <cstdint>#include <optional>#include <vector>#include "pulsatrix/featurizer.hpp"#include "pulsatrix/linear_module.hpp"
Go to the source code of this file.
Classes | |
| struct | pulsatrix::JumpReLUSaeOptions |
| class | pulsatrix::JumpReLUSparseAutoencoder |
A JumpReLU sparse autoencoder: z = pre · H(pre - θ) with pre = x W_enc + b_enc, and x̂ = z W_dec + b_dec. Each latent has its own threshold θ > 0, kept as log θ. More... | |
Namespaces | |
| namespace | pulsatrix |
The JumpReLU sparse autoencoder (FEAT-4; Rajamanoharan et al., "Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders", arXiv 2407.14435): a learned threshold per latent, trained against an L0 penalty through straight-through estimators.