graphsim
graphsim: An R package for simulating gene expression data from graph structures of biological pathways - Published in JOSS (2020)
fseval
fseval: A Benchmarking Framework for Feature Selection and Feature Ranking Algorithms - Published in JOSS (2022)
DBMS-Benchmarker
DBMS-Benchmarker: Benchmark and Evaluate DBMS in Python - Published in JOSS (2022)
sdmbench
sdmbench: R package for benchmarking species distribution models - Published in JOSS (2018)
serverless-benchmarks
SeBS: serverless benchmarking suite for automatic performance analysis of FaaS platforms.
perun
Perun is a Python package that measures the energy consumption of your applications.
pythae
Unifying Variational Autoencoder (VAE) implementations in Pytorch (NeurIPS 2022)
benchexec
BenchExec: A Framework for Reliable Benchmarking and Resource Measurement
the-next-decade-of-seismic-oceanography
This repository contains supporting information for the article The Next Decade of Seismic Oceanography: Possibilities, Challenges and Solutions (doi: 10.3389/fmars.2022.736693)
rapidae
Explore, compare and develop autoencoder models with a back-end agnostic framework
https://github.com/sintel-dev/orion
Unsupervised time series anomaly detection library
https://github.com/microsoft/mlos
MLOS is a project to enable autotuning for systems.
geoschem-gcpy
Python toolkit for GEOS-Chem. Contains basic plotting scripts, plus the suite of GEOS-Chem benchmarking utilities.
rliable
[NeurIPS'21 Outstanding Paper] Library for reliable evaluation on RL and ML benchmarks, even with only a handful of seeds.
immuneml
immuneML is a platform for machine learning analysis of adaptive immune receptor repertoire data.
https://github.com/cda-tum/mnt-bench
MNT Bench - An MNT tool for Benchmarking FCN circuits
https://github.com/ant-research/easytemporalpointprocess
EasyTPP: Towards Open Benchmarking Temporal Point Processes
https://github.com/salesforce/merlion
Merlion: A Machine Learning Framework for Time Series Intelligence
https://github.com/cleverhans-lab/cleverhans
An adversarial example library for constructing attacks, building defenses, and benchmarking both
are-we-fast-yet
Are We Fast Yet? Comparing Language Implementations with Objects, Closures, and Arrays
opencl-benchmark
A small OpenCL benchmark program to measure peak GPU/CPU performance.
bars
BARS: Towards Open Benchmarking for Recommender Systems https://openbenchmark.github.io/BARS
https://github.com/time-series-machine-learning/tsml-eval
Evaluation tools for time series machine learning algorithms.
ilamb
Python software used in the International Land Model Benchmarking (ILAMB) project
geoconv
A Python library for end-to-end learning on surfaces. It implements pre-processing functions that include geodesic algorithms, neural network layers that operate on surfaces, visualization tools and benchmarking functionalities.
specificityplus
👩💻 Code for the ACL paper "Detecting Edit Failures in LLMs: An Improved Specificity Benchmark"
TSPred
TSPred Package for R : Framework for Nonstationary Time Series Prediction
https://github.com/askabalan/jax-hpc-profiler
A convinient python script to plot performance graphs for HPC JAX codes
https://github.com/cvanaret/nonconvex_solver_comparison
This repo collects results of nonlinear optimization solvers on standard benchmark problems
https://github.com/chakib-belgaid/energy_ghz
a custom fork of gRPC benchmarking and load testing tool
ibc-benchmarking
This repository contains the performance evaluation tool and the dataset from the DSN '23 research paper "Analyzing the Performance of the Inter-Blockchain Communication Protocol".
theodolite
Theodolite is a framework for benchmarking the horizontal and vertical scalability of cloud-native applications.
https://github.com/cqcl/qvtsim
Methods for simulating quantum volume tests and comparing confidence interval constructions.
https://github.com/awslabs/barometer
A tool to automate analytic platform evaluations. Barometer helps customers to get data points needed for service selection/service configurations for given workload
https://github.com/a-herzog/tinyfastsimulator-java
Simple discrete event-oriented simulator for G/G/c models (Java edition)
qcml
A benchmarking library for quantum and classical machine learning, with specialized support for evaluating kernel methods.
ontolearner
OntoLearner: A Modular Python Library for Ontology Learning with LLMs https://pypi.org/project/OntoLearner/
timeeval-gui
[Read-Only Mirror] Benchmarking Toolkit for Time Series Anomaly Detection Algorithms using TimeEval and GutenTAG
https://github.com/rentruewang/bocoel
Bayesian Optimization as a Coverage Tool for Evaluating LLMs. Accurate evaluation (benchmarking) that's 10 times faster with just a few lines of modular code.
msaedb
Difference Benchmarking for Multivariate Small Area Estimation (https://rdocumentation.org/packages/msaeDB/versions/0.2.1)
foundation-model-benchmarking-tool
Foundation model benchmarking tool. Run any model on any AWS platform and benchmark for performance across instance type and serving stack options.
https://github.com/biodataanalysisgroup/synth4bench
A framework for generating synthetic genomics data for the evaluation of tumor-only somatic variant calling algorithms.
pyrddlgym
A toolkit for auto-generation of OpenAI Gym environments from RDDL description files.
https://github.com/cluebbers/using_r_for_hpda
Exploring R for high-performance data analytics, including memory management, GPU computing, parallel processing, benchmarks, case studies, and comparisons with Python.
https://github.com/athenarc/timevizbench
An interactive evaluation platform for scalable time series visualization methods across performance and accuracy dimensions.
quafel
Benchmarking the performance of quantum simulators in a multiprocessing fashion.
https://github.com/fernandezfran/bmxfc
:bar_chart::bulb: Datasets, pipelines and predictions of a metric for benchmarking an extreme fast-charging of Li-ion battery electrode materials
jube_cli
The JUBE benchmarking environment provides a script based framework to easily create benchmark sets, run those sets on different computer systems and evaluate the results. It is actively developed by the Jülich Supercomputing Centre of Forschungszentrum Jülich, Germany.
https://github.com/aksw/lsq
Linked SPARQL Queries (LSQ): Framework for RDFizing triple store (web) logs and performing SPARQL query extraction, analysis and benchmarking in order to produce datasets of Linked SPARQL Queries
champkit
Benchmarking toolkit for patch-based histopathology image classification.
https://github.com/biomedsciai/gene-benchmark
Benchmark gene representations from different model families
metriq-gym
metriq-gym is a framework for implementing and running standard quantum benchmarks on different quantum devices by different providers
extremeweatherbench
Benchmarking of machine learning and numerical weather prediction (MLWP & NWP) models, with a focus on extreme events.