sparse_autoencoder.hpp File Reference

Sparse autoencoder – reconstructs an activation through an overcomplete, L1-penalized hidden layer, decomposing it into a larger, sparser basis. More...

#include <cmath>
#include <cstdint>
#include <random>
#include <stdexcept>
#include <string>
#include <vector>
#include "pulsatrix/determinism.hpp"
#include "pulsatrix/linear_module.hpp"
#include "pulsatrix/mse_loss.hpp"
#include "pulsatrix/relu_module.hpp"

Include dependency graph for sparse_autoencoder.hpp:

Go to the source code of this file.

Classes
class	pulsatrix::SparseAutoencoder
	A sparse autoencoder (SAE): LinearModule(dim, hidden_dim) -> ReluModule -> LinearModule(hidden_dim, dim), trained with MSELoss to reconstruct its own input while an L1 penalty on the hidden ReLU activation pushes most hidden units to zero on any given example. More...

Namespaces
namespace	pulsatrix

Detailed Description

Sparse autoencoder – reconstructs an activation through an overcomplete, L1-penalized hidden layer, decomposing it into a larger, sparser basis.