|
pulsatrix 1.1.0
|
How similar two layers' representations of the same inputs are: linear CKA, RSA and SVCCA. More...


Go to the source code of this file.
Classes | |
| struct | pulsatrix::CkaMatrix |
CKA between every pair from two lists of representations, such as the layers of two models: values[i * cols + j] compares a[i] with b[j]. More... | |
| struct | pulsatrix::Rdm |
| A representational dissimilarity matrix: the distance between every pair of n inputs. More... | |
| struct | pulsatrix::RsaTestResult |
| RsaPermutationTest()'s result. More... | |
| struct | pulsatrix::SvccaOptions |
| Svcca()'s settings. More... | |
| struct | pulsatrix::SvccaResult |
| Svcca()'s result. More... | |
Namespaces | |
| namespace | pulsatrix |
Enumerations | |
| enum class | pulsatrix::CkaEstimator { pulsatrix::Debiased , pulsatrix::Biased } |
| Which estimator of CKA to use. More... | |
| enum class | pulsatrix::RdmMetric { pulsatrix::Correlation , pulsatrix::Euclidean , pulsatrix::Cosine } |
| How RepresentationalDissimilarity() measures the distance between two inputs. More... | |
| enum class | pulsatrix::RdmComparison { pulsatrix::Spearman , pulsatrix::Pearson } |
| How CompareRdms() correlates two RDMs. More... | |
Functions | |
| double | pulsatrix::LinearCka (const Tensor &x, const Tensor &y, CkaEstimator estimator=CkaEstimator::Debiased) |
| Linear CKA between two representations of the same inputs. | |
| CkaMatrix | pulsatrix::LinearCkaMatrix (const std::vector< Tensor > &a, const std::vector< Tensor > &b, CkaEstimator estimator=CkaEstimator::Debiased, std::size_t max_gram_bytes=std::size_t{2}<< 30) |
LinearCka() between every representation in a and every one in b. | |
| Rdm | pulsatrix::RepresentationalDissimilarity (const Tensor &x, RdmMetric metric=RdmMetric::Correlation) |
| The RDM of a representation, shape (n, ...): a row per input, later dimensions flattened. | |
| double | pulsatrix::CompareRdms (const Rdm &a, const Rdm &b, RdmComparison comparison=RdmComparison::Spearman) |
| The correlation between two RDMs of the same inputs, in the same order. | |
| double | pulsatrix::Rsa (const Tensor &x, const Tensor &y, RdmMetric metric=RdmMetric::Correlation, RdmComparison comparison=RdmComparison::Spearman) |
| RSA between two representations of the same inputs: CompareRdms(RepresentationalDissimilarity(x), RepresentationalDissimilarity(y)). | |
| RsaTestResult | pulsatrix::RsaPermutationTest (const Rdm &a, const Rdm &b, RdmComparison comparison=RdmComparison::Spearman, int64_t permutations=999, uint64_t seed=0) |
| Whether two RDMs agree more than chance: the Mantel test RSA uses. | |
| std::vector< double > | pulsatrix::CanonicalCorrelations (const Tensor &x, const Tensor &y) |
| The canonical correlations between two representations of the same inputs, largest first: min(rank x, rank y) values between 0 and 1. | |
| SvccaResult | pulsatrix::Svcca (const Tensor &x, const Tensor &y, const SvccaOptions &options={}) |
| SVCCA: CanonicalCorrelations() between each representation's top singular directions. | |
How similar two layers' representations of the same inputs are: linear CKA, RSA and SVCCA.
Centered kernel alignment (Kornblith et al., ICML 2019) compares two representations of the same n inputs, for example one layer of two models, two layers of one model, a model before and after fine-tuning, or the same layer computed on two backends. It is 1 when one representation is a rotation and uniform scaling of the other, and near 0 when they are unrelated. Unlike comparing activations directly, it doesn't need the layers to have the same width or their neurons to line up.
The debiased estimator (Song et al., JMLR 2012; Nguyen et al., ICLR 2021) is the default. The original, biased one is pulled towards 1 when there are few inputs for the width: two random, unrelated representations of 50 inputs with 1,000 features each score 0.95 with it and about 0 debiased.
Two older measures are here too, for comparison with published results that use them: