Jury
Jury: A Comprehensive Evaluation Toolkit - Published in JOSS (2024)
TorchMetrics - Measuring Reproducibility in PyTorch
TorchMetrics - Measuring Reproducibility in PyTorch - Published in JOSS (2022)
avalanche-lib
Avalanche: an End-to-End Library for Continual Learning based on PyTorch.
pytorch-ignite
High-level library to help with training and evaluating neural networks in PyTorch flexibly and transparently.
https://github.com/prometheus/prometheus
The Prometheus monitoring system and time series database.
aac-metrics
Metrics for evaluating Automated Audio Captioning systems, designed for PyTorch.
https://github.com/influxdata/telegraf
Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
https://github.com/cheind/py-motmetrics
:bar_chart: Benchmark multiple object trackers (MOT) in Python
https://github.com/chaoss/grimoirelab
GrimoireLab: platform for software development analytics and insights
yegor256/cam
Classes and Metriсs (CaM): a dataset of Java classes from public open-source GitHub repositories
augur
Python library and web service for Open Source Software Health and Sustainability metrics & data collection. You can find our documentation and new contributor information easily here: https://oss-augur.readthedocs.io/en/main/
csrankings
A web app for ranking computer science departments according to their research output in selective venues, and for finding active faculty across a wide range of areas.
jsmetrics
A Python package of algorithms and metrics used to characterise and identify jet streams, based on xarray.
https://github.com/graphite-project/graphite-web
A highly scalable real-time graphing system
pysodmetrics
PySODMetrics: A Simple and Efficient Implementation of Grayscale/Binary Segmentation Metrcis
werpy
🐍📦 Ultra-fast Python package for calculating and analyzing the Word Error Rate (WER). Built for the scalable evaluation of speech and transcription accuracy.
https://github.com/ashvardanian/simsimd
Up to 200x Faster Dot Products & Similarity Metrics — for Python, Rust, C, JS, and Swift, supporting f64, f32, f16 real & complex, i8, and bit vectors using SIMD for both AVX2, AVX-512, NEON, SVE, & SVE2 📐
galpynostatic
:zap::battery: A Python/C++ package with physics-based and data-driven models to predict optimal conditions for fast-charging lithium-ion batteries
https://github.com/pyrra-dev/pyrra
Making SLOs with Prometheus manageable, accessible, and easy to use for everyone!
https://github.com/graphite-project/whisper
Whisper is a file-based time-series database format for Graphite.
s_jsd-multilingual-bias
Code and data for the paper "An Information-Theoretic Approach and Dataset for Probing Gender Stereotypes in Multilingual Masked Language Models" (Findings of NAACL 2022)
https://github.com/cqfn/jpeek
Hosted and command-line calculator of cohesion metrics for Java code
https://github.com/chaoss/wg-dei
CHAOSS Working Group focused on Diversity, Equity, and Inclusion metrics
https://github.com/chaoss/grimoirelab-tutorial
Tutorial for installing, using, developing GrimoireLab
https://github.com/baggepinnen/spectraldistances.jl
Measure the distance between two spectra/signals using optimal transport and related metrics
https://github.com/metrics-js/process
Module for collecting different process and system metrics and providing them as a metric stream
CNVMetrics
R Package to compare copy number variant (CNV) results from multiple samples/methods
https://github.com/graphite-project/carbon
Carbon is one of the components of Graphite, and is responsible for receiving metrics over the network and writing them down to disk using a storage backend.
https://github.com/slanatech/swagger-stats
API Observability. Trace API calls and Monitor API performance, health and usage statistics in Node.js Microservices.
https://github.com/nixtla/tsfeatures
Calculates various features from time series data. Python implementation of the R package tsfeatures.
https://github.com/copyleftdev/kukai
KūKai is a modular, high-performance load-testing framework for TCP-based protocols. Inspired by the Hawaiian god Kūkailimoku (often called Kū), associated with warfare and strategic battles, KūKai aims to help you “wage war” on servers to test their capacity and resilience.
worldometer
Get live, population, geography, projected, and historical data from around the world 🌍
contrib
A python package for making stacked area plots of contributions over time.
https://github.com/chaoss/metrics
Implementation-agnostic metrics for assessing open source community health. Maintained by the CHAOSS Metrics Committee.
https://github.com/intuit/metriks
Python package of commonly used metrics for evaluating information retrieval models.
https://github.com/yegor256/plum
Programming language ultimate metrics (PLUM) collected automatically from GitHub, Google Scholar, Twitter, etc.
https://github.com/crypdick/precision-recall-gain
Precision-recall-gain for Python
https://github.com/chenliu-1996/gan-evaluator
A pip-installable evaluator for GANs (IS and FID). Accepts either dataloaders or individual batches. Supports on-the-fly evaluation during training. A working DCGAN SVHN demo script provided.
metrics_for_individual_tree_mortality_models
How performance metric choice influences individual tree mortality model selection - code and data
https://github.com/amazon-science/abstractive-factual-tradeoff
Code and data for the Dreyer et al (2023) paper on abstractiveness and factuality in abstractive summarization
https://github.com/alpaka-group/bactria
Broadly Applicable C++ Tracing and Instrumentation API :camel:
MDPIexploreR
A simple package to explore MDPI´s articles by journal. A series of functions help to obtain lists of papers, obtain data from them (turnaround times, special issues and articles types) and create summary graphs.
eco-ci-energy-estimation
Eco CI Energy estimation for Github Actions Runner VMs
https://github.com/256dpi/mgots
a wrapper for mgo that turns MongoDB into a time series database
https://github.com/brainlesion/panoptica
panoptica -- instance-wise evaluation of 3D semantic and instance segmentation maps
rafmetrics
Metrics for evaluating the performance and complexity of computer programs
dodo-database-service
This repository contains a list of KPIs and metrics used to calculate the current state of DevOps of a team/organization, based on their GitHub usage.
https://github.com/metrics-js/daemon
Daemon for collecting metrics over misc network protocols. Provides a stream for further piping of metrics.
https://github.com/prismelabs/analytics
High-perfomance, self-hosted and privacy-focused web analytics service.
https://github.com/greptimeteam/greptimedb
Open-source, cloud-native, unified observability database for metrics, logs and traces, supporting SQL/PromQL/Streaming. Available on GreptimeCloud.
https://github.com/fernandezfran/bmxfc
:bar_chart::bulb: Datasets, pipelines and predictions of a metric for benchmarking an extreme fast-charging of Li-ion battery electrode materials
cer-beans
CER ⋅ BEANS: Community Engaged Research Balanced Expressions and Assessments with Nuanced Scores
eosc-recommender-metrics
A framework for evaluating Recommender Systems (EOSC Recommender System)
https://github.com/pcpratheesh/golang-prometheus-example
A sample code snippet to learn how can connect with prometheus and monitor logs
nlg-metricverse
[COLING22] An End-to-End Library for Evaluating Natural Language Generation
Colocalization
Colocalization metrics and distances for images or their sparse representations.
https://github.com/metrics-js/metric
The metric class definition which metric objects is instansiated from.
comparative-embedding-visualization
A Jupyter widget for comparing two embeddings with shared labels by their confusion, neighborhoods, and size.