Latent Scaling (Minder et al., Eq. 4): how much of latent j's b-side decoder direction each source actually needs. For a target Y^m over the inputs where it's computed, β^m = ⟨Y^m d, f⟩ / (|f|² |d|²) with d = d_j^b, the best scale of f d against Y^m.
More...
#include <crosscoder.hpp>
Latent Scaling (Minder et al., Eq. 4): how much of latent j's b-side decoder direction each source actually needs. For a target Y^m over the inputs where it's computed, β^m = ⟨Y^m d, f⟩ / (|f|² |d|²) with d = d_j^b, the best scale of f d against Y^m.
- Error targets are what the rest of the crosscoder leaves:
x^m - (x̂^m - f_j d_j^m). A b-only latent whose ν_error is high is complete shrinkage: source a needs it too, but the sparsity penalty zeroed its a decoder.
- Reconstruction targets are
x̂^m. A high ν_reconstruction is latent decoupling: the same thing is also written to source a by other latents.
- ν = β^a / β^b. Minder et al. call a latent b-specific when ν_reconstruction < 0.5 and ν_error < 0.2; a negative β^b leaves ν meaningless, and they drop such latents.
◆ active
| int64_t pulsatrix::LatentScaling::active = 0 |
Inputs where the latent fired.
◆ beta_error_a
| double pulsatrix::LatentScaling::beta_error_a = 0 |
◆ beta_error_b
| double pulsatrix::LatentScaling::beta_error_b = 0 |
◆ beta_reconstruction_a
| double pulsatrix::LatentScaling::beta_reconstruction_a = 0 |
◆ beta_reconstruction_b
| double pulsatrix::LatentScaling::beta_reconstruction_b = 0 |
◆ latent
| int64_t pulsatrix::LatentScaling::latent = 0 |
◆ nu_error
| double pulsatrix::LatentScaling::nu_error = 0 |
◆ nu_reconstruction
| double pulsatrix::LatentScaling::nu_reconstruction = 0 |
The documentation for this struct was generated from the following file: