pulsatrix 1.1.0
Loading...
Searching...
No Matches
fitness_metrics.hpp File Reference

ProteinGym's metrics for how well predicted variant scores track measured fitness (PLM-3), computed exactly as ProteinGym's performance_DMS_benchmarks.py does. More...

#include <vector>
Include dependency graph for fitness_metrics.hpp:
This graph shows which files directly or indirectly include this file:

Go to the source code of this file.

Classes

struct  pulsatrix::FitnessMetrics
 All five, as ProteinGym reports them. More...
 

Namespaces

namespace  pulsatrix
 

Functions

double pulsatrix::SpearmanCorrelation (const std::vector< double > &x, const std::vector< double > &y)
 Spearman's rank correlation, ties given their average rank (scipy's spearmanr). NaN if either input is constant.
 
double pulsatrix::RocAuc (const std::vector< int > &labels, const std::vector< double > &scores)
 Area under the ROC curve of scores for labels (0 or 1), ties counted half (sklearn's roc_auc_score). NaN if only one class is present.
 
double pulsatrix::MatthewsCorrelation (const std::vector< int > &labels, const std::vector< int > &predicted)
 Matthews correlation of binary labels and predicted (sklearn's matthews_corrcoef); 0 when a row or column of the confusion matrix is empty.
 
double pulsatrix::MedianSplitMcc (const std::vector< int > &labels, const std::vector< double > &scores)
 ProteinGym's MCC: scores at or above their median predict 1.
 
double pulsatrix::Ndcg (const std::vector< double > &true_scores, const std::vector< double > &predicted, double top_percent=10.0)
 ProteinGym's NDCG: gains are the true scores min-max scaled to [0, 1]; the variants the model ranks in the top top_percent (floor of the count) contribute gain / log2(rank + 1), normalized by the same sum for the truly best ones. Zero gains are dropped.
 
double pulsatrix::TopRecall (const std::vector< double > &true_scores, const std::vector< double > &predicted, double top_true=10.0, double top_model=10.0)
 ProteinGym's top recall: of the variants at or above the true scores' (100 - top) percentile, the fraction also at or above the predicted scores' percentile. Percentiles interpolate linearly, as numpy's do.
 
double pulsatrix::Percentile (std::vector< double > values, double q)
 The q-th percentile (0 to 100) with linear interpolation (numpy's default).
 
FitnessMetrics pulsatrix::EvaluateFitness (const std::vector< double > &dms_scores, const std::vector< int > &dms_bins, const std::vector< double > &predicted)
 FitnessMetrics for one assay: measured scores and their binarized labels, and predictions.
 

Detailed Description

ProteinGym's metrics for how well predicted variant scores track measured fitness (PLM-3), computed exactly as ProteinGym's performance_DMS_benchmarks.py does.