Data

Tools

Indexes

Applications

Experiments

Open Source Science

Recent Releases of PerMetrics

PerMetrics - Resolve problems pointed out by reviewers of JOSS journal

  • Update documents, examples, and README
  • Update pre-condition check for all metrics
  • Remove decimal parameter in all metrics
  • Replace non_zero, positive parameters by force_finite, finite_value in many metrics.
  • Add features tests and comparisons tests

Scientific Software - Peer-reviewed - Python
Published by thieu1995 almost 2 years ago

PerMetrics -

  • Add metrics to ClusteringMetric class:
    • Mean Squared Error Index (MSEI)
    • Adjusted Rand Score (ARS)
    • Gamma Score (GAS)
    • Gplus Score (GPS)
  • Speed up all metrics in ClusteringMetric by transferring the computation to cluster_util module.
  • Add speed up examples for clustering metrics (10x-100x faster than previous version)
  • Update documents, ChangeLog, Readme

Scientific Software - Peer-reviewed - Python
Published by thieu1995 over 2 years ago

PerMetrics - PerMetrics: A Framework of Performance Metrics for Machine Learning Models

  • Reformat ClassificationMetric, RegressionMetric, and ClusteringMetric
  • Fix bugs in metrics:
    • ROC-AUC score
    • F2 score, FBeta score
    • Hamming Score, Accuracy Score
  • Add metrics to ClassificationMetric:
    • Brier Score Loss (BSL)
    • Kullback-Leibler Divergence Loss (KLDL)
    • Cross-Entropy Loss (CEL)
    • Hinge Loss (HL)
  • Update docs, examples

Scientific Software - Peer-reviewed - Python
Published by thieu1995 over 2 years ago

PerMetrics -

  • Fix bug in get_support() function
  • Fix bug rounding number in ClassificationMetric
  • Update logo and docs

Scientific Software - Peer-reviewed - Python
Published by thieu1995 over 2 years ago

PerMetrics -

  • Remove all lowercase shortname of all metrics
  • Fix bugs in GINI function belongs to ClassificationMetric
  • Fix bugs in some functions belong to ClusteringMetric
  • Update characteristics of most of the functions (valid ranges and best value)
  • Add metrics to ClusteringMetrics
    • Entropy Score (ES)
    • Purity Score (PuS)
    • Tau Score (TS)
    • Duda-Hart Index (DHI)
    • Sum of Squared Error Index (SSEI)
    • Beale Index (BI)
    • R-Squared Index (RSI)
    • Density-Based Clustering Validation Index (DBCVI)
    • Hartigan Index (HI)
  • Add get_support() method in RegressionMetric, ClassificationMetric, and ClusteringMetric
  • Update and add more examples to all classes.
  • Update documents for all metrics.

Scientific Software - Peer-reviewed - Python
Published by thieu1995 over 2 years ago

PerMetrics -

  • Add ClusteringMetric:
    • Add internal metrics (Need X features and y_pred)
    • ballhallindex (BHI)
    • calinskiharabaszindex (CHI)
    • xiebeniindex (XBI)
    • banfeldrafteryindex (BRI)
    • daviesbouldinindex (DBI)
    • detratioindex (DRI)
    • dunn_index (DI)
    • ksqdetwindex (KDI)
    • logdetratio_index (LDRI)
    • logssratio_index (LSRI)
    • silhouette_index (SI)
    • Add external metrics (Need ytrue and ypred)
    • mutualinfoscore (MIS)
    • normalizedmutualinfo_score (NMIS)
    • rand_score (RaS)
    • fowlkesmallowsscore (FMS)
    • homogeneity_score (HS)
    • completeness_score (CS)
    • vmeasurescore (VMS)
    • precision_score (PrS)
    • recall_score (ReS)
    • fmeasurescore (FmS)
    • czekanowskidicescore (CDS)
    • hubertgammascore (HGS)
    • jaccard_score (JS)
    • kulczynski_score (KS)
    • mcnemarscore (MNS)
    • phi_score (PhS)
    • rogerstanimotoscore (RTS)
    • russelraoscore (RRS)
    • sokalsneath1score (SS1S)
    • sokalsneath2score (SS2S)
  • Add examples for ClusteringMetric
  • Add LabelEncoder class

Scientific Software - Peer-reviewed - Python
Published by thieu1995 over 2 years ago

PerMetrics -

Update

Update ClassificationMetric: + Rename confusion_matrix() in util file + Support binary and multi-class classification with one-hot-encoder format + Add Cohen's Kappa score + Add Jaccard Similarity Index (Jaccard similarity coefficient) + Add G-mean score + Add GINI index + Add ROC-AUC metric

Scientific Software - Peer-reviewed - Python
Published by thieu1995 over 2 years ago

PerMetrics -

Update

Add regression metrics: + covariance (COV): https://corporatefinanceinstitute.com/resources/data-science/covariance/ + correlation (COR): https://corporatefinanceinstitute.com/resources/data-science/covariance/ + efficiency coefficient (EC): https://doi.org/10.1016/j.solener.2019.01.037 + overall index (OI): https://doi.org/10.1016/j.solener.2019.01.037 + coefficient of residual mass (CRM): https://doi.org/10.1016/j.solener.2019.01.037

Scientific Software - Peer-reviewed - Python
Published by thieu1995 almost 3 years ago

PerMetrics -

Update

  • Fix bug in MCC metric
  • Fix bug exit() program
  • Update documents
  • Update examples

Scientific Software - Peer-reviewed - Python
Published by thieu1995 about 3 years ago

PerMetrics -

Update

Add classification metrics: + Add Precision Score (PS) + Add Negative Predictive Value (NPV) + Add Recall Score (RS) + Add Accuracy Score (AS) + Add F1 Score (F1S) + Add F2 Score (F2S) + Add F Beta Score (FBS) + Add Specificity Score (SS) + Add Matthews Correlation Coefficient (MCC) + Add Hamming Loss (HL) + Add Lift Score (LS)

  • Update reference documents
  • Update examples

Scientific Software - Peer-reviewed - Python
Published by thieu1995 over 3 years ago

PerMetrics -

Version 1.2.2

Update

  • Add Absolute R metric (AR)
  • Add A30 metric (A30)
  • Add Mean Bias Error (MBE)
  • Add Mean Percentage Error (MPE)
  • Add Adjusted R2 (AR2)
  • Update reference documents
  • Update examples

Scientific Software - Peer-reviewed - Python
Published by thieu1995 over 3 years ago

PerMetrics -

Update

  • Fix bugs copy original data from Evaluator class
  • Fix bugs of some methods in Regression class
  • Add utils module
  • Update reference documents
  • Update examples

Scientific Software - Peer-reviewed - Python
Published by thieu1995 over 3 years ago

PerMetrics -

Change models

  • Add Evaluator class, the base class for all metrics
  • Rename class Metrics in regression to RegressionMetric
  • Rename class Metrics in classification to ClassificationMetric
  • Update input parameters in both RegressionMetric and ClassificationMetric
  • Update input parameters in all metric functions
  • Add docstring for all methods
  • Merge all singleloss metrics to RegressionMetric
  • Update examples
  • Add new documents and website

Scientific Software - Peer-reviewed - Python
Published by thieu1995 almost 4 years ago

PerMetrics -

  • Fix bugs of R2S, MAPE and some other functions

Scientific Software - Peer-reviewed - Python
Published by thieu1995 over 4 years ago

PerMetrics -

Scientific Software - Peer-reviewed - Python
Published by thieu1995 over 4 years ago

PerMetrics -

Scientific Software - Peer-reviewed - Python
Published by thieu1995 over 4 years ago

PerMetrics -

Scientific Software - Peer-reviewed - Python
Published by thieu1995 over 4 years ago

PerMetrics -

Scientific Software - Peer-reviewed - Python
Published by thieu1995 almost 5 years ago

PerMetrics -

Change models

  • Change class Metrics in regression.py to take both OOP style and Functional style.
  • Add some new metrics including
  • PCD: Prediction of Change in Direction
  • E: Entropy
  • CE: Cross Entropy
  • KLD: Kullback Leibler Divergence
  • JSD: Jensen Shannon Divergence
  • VAF: Variance Accounted For
  • RAE: Relative Absolute Error
  • A10: A10 Index
  • A20: A20 Index
  • NRMSE: Normalized root Mean Square Error

Change others

  • Examples:
    • Update examples of old metrics
    • Add examples for new metrics

Scientific Software - Peer-reviewed - Python
Published by thieu1995 almost 5 years ago

PerMetrics -

Change models

  • Add methods

    • 21. Gini coefficient
      • Based on other code (Mathlab code)
      • Based on wiki version
    • 22. Mean Log Likelihood

  • Add new class SingleLoss for numbers-verse-numbers, the output is numbers (not single number as regression.py)

    • 1. Relative error
    • 2. Absolute error
    • 3. Squared error
    • 4. Squared log error
    • 5. Log likelihood

  • Add new class for classification metrics

    • 1. Mean Log Likelihood

Change others

  • Examples:
    • Add examples
    • Add documents

Scientific Software - Peer-reviewed - Python
Published by thieu1995 over 5 years ago

PerMetrics -

Scientific Software - Peer-reviewed - Python
Published by thieu1995 over 5 years ago

PerMetrics -

Change models

  • Add methods
    • (Pearson’s Correlation Index)**2 or R2s (or R2)
    • Kling-Gupta Efficiency (KGE)
    • Deviation of Runoff Volume (DRV) ### Change others
  • Examples:
    • Add examples
    • Add documents

Scientific Software - Peer-reviewed - Python
Published by thieu1995 over 5 years ago

PerMetrics -

Change models

  • Add method for multiple metrics called at the same time
    • Call by list of function names (default function parameters)
    • Call by list of function names and parameters (can change function parameters) ### Change others
  • Examples:
    • Add all examples for all metrics
    • Add example for multiple metrics called at the same time
  • Documents: Add all documents for all metrics

Scientific Software - Peer-reviewed - Python
Published by thieu1995 over 5 years ago

PerMetrics - First release

For Regression models, we have 17 metrics

  1. Explained Variance Score
  2. Max Error
  3. Mean Absolute Error
  4. Mean Squared Error
  5. Root Mean Squared Error
  6. Mean Squared Log Error
  7. Median Absolute Error
  8. Mean Relative Error
  9. Mean Absolute Percentage Error
  10. Symmetric Mean Absolute Percentage Error
  11. Mean Arctangent Absolute Percentage Error
  12. Mean Absolute Scaled Error
  13. Nash-Sutcliffe Efficiency Coefficient
  14. Willmott Index
  15. Pearson’s Correlation Index
  16. Confidence Index
  17. Coefficient of Determination

Scientific Software - Peer-reviewed - Python
Published by thieu1995 over 5 years ago