Projects | Open Source Science

Scientific Software

Updated 6 months ago

imodels — Peer-reviewed • Rank 21.0 • Science 100%

imodels: a python package for fitting interpretable models - Published in JOSS (2021)

ai artificial-intelligence bayesian-rule-list data-science explainable-ai explainable-ml imodels interpretability machine-learning ml optimal-classification-tree python rule-learning rulefit rules scikit-learn statistics supervised-learning

Engineering Economics

Scientific Software · Peer-reviewed

Scientific Software

Updated 6 months ago

modelStudio — Peer-reviewed • Rank 13.5 • Science 95%

modelStudio: Interactive Studio with Explanations for ML Predictive Models - Published in JOSS (2019)

ai explainable explainable-ai explainable-machine-learning explanatory-model-analysis human iml interactive interactivity interpretability interpretable interpretable-machine-learning learning machine model model-visualization r visualization xai

Engineering (40%) Earth and Environmental Sciences (40%)

Scientific Software · Peer-reviewed

Scientific Software

Updated 6 months ago

FAT Forensics — Peer-reviewed • Rank 9.9 • Science 95%

FAT Forensics: A Python Toolbox for Implementing and Deploying Fairness, Accountability and Transparency Algorithms in Predictive Systems - Published in JOSS (2020)

accountability explainability explainable-ai fairness interpretability interpretable-ai machine-learning transparency

Mathematics Engineering

Scientific Software · Peer-reviewed

Scientific Software

Updated 6 months ago

ExpFamilyPCA.jl — Peer-reviewed • Rank 4.5 • Science 100%

ExpFamilyPCA.jl: A Julia Package for Exponential Family Principal Component Analysis - Published in JOSS (2025)

compression compression-algorithm denoising dimensionality-reduction epca exponential-family interpretability julia machine-learning pca principal-component-analysis reinforcement-learning signal-processing

Medicine (40%)

Scientific Software · Peer-reviewed

Scientific Software

Updated 6 months ago

TSInterpret — Peer-reviewed • Rank 7.2 • Science 93%

TSInterpret: A Python Package for the Interpretability of Time Series Classification - Published in JOSS (2023)

counterfactual-explanations explainable-ai explainable-artificial-intelligence explainable-ml feature-attribution interpretability interpretable-machine-learning interpretable-ml time-series

Engineering (40%) Sociology (40%)

Scientific Software · Peer-reviewed

Updated 6 months ago

tf-explain • Rank 18.9 • Science 77%

Interpretability Methods for tf.keras models with Tensorflow 2.x

deep-learning interpretability keras machine-learning tensorflow tf2 visualization

Updated 6 months ago

quantus • Rank 16.1 • Science 77%

Quantus is an eXplainable AI toolkit for responsible evaluation of neural network explanations

deep-learning explainable-ai interpretability machine-learning pytorch quantification-evaluation-methods reproducibility tensorflow xai

Updated 6 months ago

captum • Rank 27.9 • Science 64%

Model interpretability and understanding for PyTorch

feature-attribution feature-importance interpretability interpretable-ai interpretable-ml

Updated 6 months ago

alibi • Rank 20.9 • Science 67%

Algorithms for explaining machine learning models

counterfactual explanations interpretability machine-learning xai

Updated 6 months ago

interpret • Rank 26.4 • Science 59%

Fit interpretable models. Explain blackbox machine learning.

ai artificial-intelligence bias blackbox differential-privacy explainability explainable-ai explainable-ml gradient-boosting iml interpretability interpretable-ai interpretable-machine-learning interpretable-ml interpretml machine-learning scikit-learn transparency xai

Updated 6 months ago

inseq • Rank 13.6 • Science 67%

Interpretability for sequence generation models 🐛 🔍

attribution-methods captum deep-learning explainable-ai generative-ai huggingface interpretability language-generation language-model large-language-models natural-language-processing sequence-to-sequence transformers

Updated 6 months ago

imodelsx • Rank 14.5 • Science 64%

Interpret text data using LLMs (scikit-learn compatible).

ai deep-learning explainability huggingface interpretability language-model machine-learning ml natural-language-processing natural-language-understanding neural-network pytorch scikit-learn text text-classification transformer-models xai

Updated 6 months ago

timbertrek • Rank 10.6 • Science 67%

Explore and compare 1K+ accurate decision trees in your browser!

decision-tree interactive-visualizations interpretability rashomon visualization

Updated 6 months ago

shap • Rank 31.6 • Science 46%

A game theoretic approach to explain the output of any machine learning model.

deep-learning explainability gradient-boosting interpretability machine-learning shap shapley

Updated 6 months ago

corelay • Rank 11.2 • Science 64%

CoRelAy is a tool to compose small-scale (single-machine) analysis pipelines.

interpretability machine-learning pipeline-framework python spectral-clustering xai

Updated 6 months ago

transformers-interpret • Rank 19.3 • Science 54%

Model explainability that works seamlessly with 🤗 transformers. Explain your transformers model in just 2 lines of code.

captum computer-vision deep-learning explainable-ai interpretability machine-learning model-explainability natural-language-processing neural-network nlp transformers transformers-model

Updated 6 months ago

torchcam • Rank 19.3 • Science 54%

Class activation maps for your PyTorch models (CAM, Grad-CAM, Grad-CAM++, Smooth Grad-CAM++, Score-CAM, SS-CAM, IS-CAM, XGrad-CAM, Layer-CAM)

activation-maps class-activation-map cnn deep-learning grad-cam gradcam gradcam-plus-plus interpretability interpretable-deep-learning python pytorch saliency-map score-cam smoothgrad

Updated 6 months ago

virelay • Rank 9.3 • Science 64%

ViRelAy is a visualization tool for the analysis of data as generated by CoRelAy.

angular flask interpretability machine-learning python visualization xai

Updated 6 months ago

yggdrasil-decision-forests • Rank 21.5 • Science 49%

A library to train, evaluate, interpret, and productionize decision forest models such as Random Forest and Gradient Boosted Decision Trees.

cart cli cpp decision-forest decision-trees distributed-computing go gradient-boosting interpretability javascript machine-learning ml pypi python random-forest tensorflow

Updated 6 months ago

grad-cam • Rank 23.6 • Science 46%

Advanced AI Explainability for computer vision. Support for CNNs, Vision Transformers, Classification, Object detection, Segmentation, Image similarity and more.

class-activation-maps computer-vision deep-learning explainable-ai explainable-ml grad-cam image-classification interpretability interpretable-ai interpretable-deep-learning machine-learning object-detection pytorch score-cam vision-transformers visualizations xai

Updated 6 months ago

multi-mode-cnn-pytorch • Rank 2.2 • Science 67%

A PyTorch implementation of the Multi-Mode CNN to reconstruct Chlorophyll-a time series in the global ocean from oceanic and atmospheric physical drivers

attention-mechanisms biogeochemical-regions chlorophyll-a climate convolutional-neural-networks deep-learning earth-observation global-scale interpretability marine-science ocean-color oceanography physical-drivers phytoplankton pytorch regression remote-sensing satellite-data seasonal-interannual-timescale time-series

Updated 6 months ago

ExplainableAI • Rank 7.7 • Science 54%

Explainable AI in Julia.

attribution-methods explainable-ai feature-attribution interpretability interpretable-ai julia lrp xai

Updated 6 months ago

cvplot • Rank 4.4 • Science 57%

Understand machine learning models with Contribution-Value plots

interpretability learning machine

Updated 6 months ago

asent • Rank 14.9 • Science 44%

Asent is a python library for performing efficient and transparent sentiment analysis using spaCy.

interpretability natural-language-processing nlp python3 sentiment-analysis spacy spacy-extensions

Updated 6 months ago

DALEX • Rank 22.6 • Science 36%

moDel Agnostic Language for Exploration and eXplanation

black-box dalex data-science explainable-ai explainable-artificial-intelligence explainable-ml explanations explanatory-model-analysis fairness iml interpretability interpretable-machine-learning machine-learning model-visualization predictive-modeling responsible-ai responsible-ml xai

Updated 5 months ago

https://github.com/csinva/tree-prompt • Rank 6.0 • Science 46%

Tree prompting: easy-to-use scikit-learn interface for improved prompting.

ai artificial-intelligence classification controllability decision-tree huggingface interpretability language-model language-models llm llm-inference machine-learning prompt-engineering prompting scikit-learn

Updated 6 months ago

hierarchical-dnn-interpretations • Rank 9.6 • Science 38%

Using / reproducing ACD from the paper "Hierarchical interpretations for neural network predictions" 🧠 (ICLR 2019)

acd ai artificial-intelligence convolutional-neural-networks data-science deep-learning deep-neural-networks explainability explainable-ai feature-importance iclr interpretability interpretation jupyter-notebook machine-learning ml neural-network python pytorch statistics

Updated 6 months ago

breakDown • Rank 13.8 • Science 33%

Model Agnostics breakDown plots

data-science iml interpretability machine-learning visual-explanations xai

Updated 6 months ago

tree-influence • Rank 8.1 • Science 31%

Influence Estimation for Gradient-Boosted Decision Trees

boostin explainability gradient-boosted-trees influence-estimation influence-functions influential-examples instance-attribution instance-based interpretability tracin

Updated 6 months ago

deeplift • Rank 17.8 • Science 20%

Public facing deeplift repo

deeplift guided-backpropagation integrated-gradients interpretability interpretable-deep-learning saliency-map sensitivity-analysis

Updated 6 months ago

iBreakDown • Rank 17.6 • Science 20%

Break Down with interactions for local explanations (SHAP, BreakDown, iBreakDown)

breakdown iml interpretability shapley xai

Updated 5 months ago

https://github.com/csinva/disentangled-attribution-curves • Rank 4.4 • Science 33%

Using / reproducing DAC from the paper "Disentangled Attribution Curves for Interpreting Random Forests and Boosted Trees"

ai artificial-intelligence boosting ensemble-model explainable-ai feature-engineering feature-importance interpretability machine-learning ml python random-forest random-forests scikit-learn statistics stats

Updated 6 months ago

https://github.com/braindatalab/xai-tris • Rank 1.8 • Science 26%

XAI-Tris

artificial-intelligence benchmarking captum explainability explainable-ai explainable-ml innvestigate interpretability interpretable-ai interpretable-ml machine-learning python xai

Updated 4 months ago

https://github.com/google-research/reverse-engineering-neural-networks • Rank 10.6 • Science 13%

A collection of tools for reverse engineering neural networks.

deep-learning interpretability machine-learning

Updated 6 months ago

live • Rank 10.4 • Science 10%

Local Interpretable (Model-agnostic) Visual Explanations - model visualization for regression problems and tabular data based on LIME method. Available on CRAN

iml interpretability lime machine-learning model-visualization visual-explanations xai

Updated 6 months ago

tree.interpreter • Rank 9.7 • Science 10%

Decision tree interpreter for randomForest/ranger as described in

data-science datascience interpretability machine-learning r random-forest

Updated 6 months ago

bioexp • Rank 8.2 • Science 10%

Explainability of Deep Learning Models

ablation activation-maximization causal-inference causality clustering concept-extraction dissection explainability explainable-ai gradcam interpretability interpretable-ai intervention joint-analysis rct uncertainty

Updated 6 months ago

coefeasy • Science 26%

Coefeasy is an R package under development for making regression coefficients more accessible. With this tool, you can read and report key coefficients instantly.

coefficients interpretability logistic-regression ols-regression statistics

Updated 6 months ago

icml19-egocnn • Science 44%

Code for "Distributed, Egocentric Representations of Graphs for Detecting Critical Structures" (ICML 2019)

convolutional-neural-networks graph-convolutional-neural-networks graph-embeddings graph-networks graph-neural-networks icml icml-2019 interpretability scale-free-networks self-similarity

Updated 6 months ago

pnet_robustness • Science 75%

Reliable interpretability of biology-inspired deep neural networks

deep-learning interpretability network-analysis neural-networks

Updated 6 months ago

symbolic-governed-mistral-artifact • Science 44%

Tier-10 sealed governance artifact for Mistral-7B with exact-match benchmarks and symbolic verifier.

alignment artifact benchmark interpretability llm mistral symbolic-governance

Updated 5 months ago

https://github.com/csinva/transformation-importance • Science 23%

Using / reproducing TRIM from the paper "Transformation Importance with Applications to Cosmology" 🌌 (ICLR Workshop 2020)

ai artificial-intelligence attribution data-science deep-learning deep-neural-networks explainability explainable-ai feature-engineering feature-importance frequency-domain interpretability interpretation machine-learning ml neural-network pytorch transform transformation wavelet-analysis

Updated 6 months ago

kgraph-ts • Science 67%

Graph Embedding for Interpretable Time Series Clustering

clustering graph graph-embedding graph-representation interpretability networkx python python3 time-series time-series-analysis time-series-clustering

Updated 6 months ago

scene-representation-diffusion-model • Science 23%

Linear probe found representations of scene attributes in a text-to-image diffusion model

explainability image-editing interpretability scene stable-diffusion

Updated 6 months ago

https://github.com/berenslab/interpretable-deep-survival-analysis • Science 36%

An interpretable end-to-end CNN for disease progression modeling that predicts late AMD onset (MICCAI 2024)

cnn coxph deep-learning disease-prediction image-based interpretability survival-analysis

Updated 6 months ago

attention-meets-perturbation • Science 67%

📝 Official Implementation of "Attention Meets Perturbation: Robust and Interpretable Attention with Adversarial Training"

adversarial-training allennlp attention-mechanism binary-classification ieee interpretability natural-language-inference natural-language-processing pytorch question-answering robustness

Updated 5 months ago

https://github.com/csinva/clinical-rule-development • Science 49%

Building and vetting clinical decision rules.

ai artificial-intelligence clinical decision-rules decision-tree emergency-medicine explainable-ai explainable-ml interpretability large-language-models llm machine-learning medical medicine ml pecarn statistics trauma xai

Updated 6 months ago

shapiq • Science 36%

Shapley Interactions and Shapley Values for Machine Learning

banzhaf-index explainability explainable-ai feature-attribution feature-importance feature-interactions game-theory interpretability interpretable-machine-learning machine-learning python shap shapiq shapley shapley-interactions

Updated 6 months ago

captum-tutorials • Science 54%

captum deep-learning interpretability neural-style-transfer style-transfer

Updated 6 months ago

contrastiveexplanation • Science 54%

Contrastive Explanation (Foil Trees), developed at TNO/Utrecht University

contrastive-explanations explainable-ai foil interpretability machine-learning model-agnostic

Updated 6 months ago

https://github.com/birkhoffg/explainable-ml-papers • Science 23%

A list of research papers of explainable machine learning.

academic awesome counterfactual-explanations explainability explainable-ml explanations human-ai-interaction human-in-the-loop human-in-the-loop-machine-learning interpretability interpretable-ml interpretable-models machine-learning paper recourse research survey trustworthy-machine-learning xai

Updated 6 months ago

understanding-clip-ood • Science 36%

Official code for the paper: "When and How Does CLIP Enable Domain and Compositional Generalization?" (ICML 2025 Spotlight)

clip interpretability machine-learning mechanistic-interpretability ood-generalization robustness vision-language-models

Updated 6 months ago

automated-brain-explanations • Science 36%

Generating and validating natural-language explanations for the brain.

ai-for-science artificial-intelligence automated-interpretability data-science explanation fmri fmri-data-analysis gpt gpt4 huggingface interpretability interpretable-embeddings language-model large-language-models machine-learning mechanistic-interpretability natural-language-processing neuroscience xai

Updated 5 months ago

https://github.com/csinva/hpa-interp • Science 13%

Interpretations on the HPA dataset.

artificial-intelligence biological-data-analysis deep-learning explainability interpretability machine-learning medical-imaging ml neural-network pytorch

Updated 5 months ago

https://github.com/csinva/iprompt • Science 36%

Finding semantically meaningful and accurate prompts.

ai autoprompt deep-learning explainability explainable-ai interpretability iprompt language-model large-language-models ml natural-language-processing neural-network prompting text text-classification xai

Updated 6 months ago

rome • Science 44%

Locating and editing factual associations in GPT (NeurIPS 2022)

gpt interpretability pytorch transformers

Updated 6 months ago

awesome-attention-heads • Science 67%

An awesome repository & A comprehensive survey on interpretability of LLM attention heads.

attention-head-mining attention-mechanism awesome chain-of-thought circuit-analysis cognitive-neuroscience gpt interpretability large-language-models llm llm-reasoning machine-psychology research-paper survey transformer visualization-tools

Updated 6 months ago

bcf-iv • Science 67%

Package for heterogeneous causal effects in the presence of imperfect compliance (e.g., instrumental variables, fuzzy regression discontinuity designs)

bayesian-statistics causal-inference fuzzy-rdd instrumental-variable interpretability interpretable-machine-learning machine-learning r statistics

Updated 4 months ago

https://github.com/kundajelab/fastism • Science 13%

In-silico Saturation Mutagenesis implementation with 10x or more speedup for certain architectures.

cnn genomics interpretability interpretable-deep-learning ism

Updated 6 months ago

https://github.com/betswish/mirage • Science 26%

Easy-to-use MIRAGE code for faithful answer attribution in RAG applications. Paper: https://aclanthology.org/2024.emnlp-main.347/

attribution-methods interpretability model-internals nlp retrieval-augmented-generation

Updated 6 months ago

awesome-production-machine-learning • Science 62%

A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning

awesome awesome-list data-mining deep-learning explainability interpretability large-scale-machine-learning large-scale-ml machine-learning machine-learning-operations ml-operations ml-ops mlops privacy-preserving privacy-preserving-machine-learning privacy-preserving-ml production-machine-learning production-ml responsible-ai

Updated 6 months ago

feud • Science 62%

AI Division, Reverse Engineering CNN Trojans

artificial-intelligence computer-vison interpretability interpretable-ai reverse-engineering

Updated 6 months ago

talktomodel • Science 67%

TalkToModel gives anyone with the powers of XAI through natural language conversations 💬!

ai conversational-ai conversationalxai dialogue-systems explainability explainable-artificial-intelligence explainable-ml explanation interpretability interpretable-machine-learning machine-learning-library machine-learning-papers ml paper-implementation papers xai

Updated 6 months ago

jeli • Science 67%

Joint Embedding-classifier Learning for improved Interpretability (JELI)

collaborative-filtering drug-repurposing factorization-machine interpretability