batch_norm_module.hpp File Reference

Batch normalization (Ioffe & Szegedy, 2015) – per-channel statistics computed across the batch and spatial dimensions jointly, unlike GroupNormModule's per-(batch-row, group) statistics. More...

#include <optional>
#include <initializer_list>
#include <vector>
#include "pulsatrix/module.hpp"

Include dependency graph for batch_norm_module.hpp:

This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Classes
class	pulsatrix::BatchNormModule
	y_{n,c,h,w} = gamma_c * (x_{n,c,h,w} - mu_c)/std_c + beta_c, mu_c/std_c computed per channel c over every (n, h, w) element jointly – BatchNorm's defining statistic, and the reason this module didn't exist before campaign_exai_dl_library_batch_dimension_support: it has nothing to compute over without a real batch dimension. Input/output are rank-4 (N, channels, H, W), the same convention Conv2DModule/GroupNormModule already establish. More...

Namespaces
namespace	pulsatrix

Detailed Description

Batch normalization (Ioffe & Szegedy, 2015) – per-channel statistics computed across the batch and spatial dimensions jointly, unlike GroupNormModule's per-(batch-row, group) statistics.