pulsatrix 1.1.0
Loading...
Searching...
No Matches
pulsatrix::JumpReLUSaeOptions Struct Reference

#include <jumprelu_sparse_autoencoder.hpp>

Public Attributes

float l0_coefficient = 1e-3f
 λ, the L0 penalty's weight: the loss is mean((x̂ - x)²) + λ * L0, L0 averaged over the batch. The paper sums the squared error over dimensions; with the mean, λ is smaller by the input dimension.
 
float bandwidth = 1e-3f
 ε, the bandwidth of the rectangle kernel that stands in for the step's derivative.
 
float initial_threshold = 1e-3f
 Every latent's starting threshold θ (the paper's 0.001).
 
uint64_t seed = 0
 

Member Data Documentation

◆ bandwidth

float pulsatrix::JumpReLUSaeOptions::bandwidth = 1e-3f

ε, the bandwidth of the rectangle kernel that stands in for the step's derivative.

◆ initial_threshold

float pulsatrix::JumpReLUSaeOptions::initial_threshold = 1e-3f

Every latent's starting threshold θ (the paper's 0.001).

◆ l0_coefficient

float pulsatrix::JumpReLUSaeOptions::l0_coefficient = 1e-3f

λ, the L0 penalty's weight: the loss is mean((x̂ - x)²) + λ * L0, L0 averaged over the batch. The paper sums the squared error over dimensions; with the mean, λ is smaller by the input dimension.

◆ seed

uint64_t pulsatrix::JumpReLUSaeOptions::seed = 0

The documentation for this struct was generated from the following file: