https://github.com/andreartelt/awesome-explainable-ai
A collection of research materials on explainable AI/ML
Science Score: 23.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
○codemeta.json file
-
○.zenodo.json file
-
✓DOI references
Found 3 DOI reference(s) in README -
✓Academic publication links
Links to: arxiv.org, researchgate.net, sciencedirect.com, springer.com, mdpi.com, ieee.org, acm.org -
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (7.1%) to scientific vocabulary
Last synced: 10 months ago
·
JSON representation
Repository
A collection of research materials on explainable AI/ML
Basic Info
Statistics
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
- Releases: 0
Fork of wangyongjie-ntu/Awesome-explainable-AI
Created over 5 years ago
· Last pushed over 5 years ago
https://github.com/andreArtelt/Awesome-explainable-AI/blob/master/
[](https://github.com/wangyongjie-ntu/Awesome-explainable-AI)
[](https://github.com/wangyongjie-ntu/Awesome-explainable-AI/graphs/commit-activity)
[](https://github.com/wangyongjie-ntu/Awesome-explainable-AI/stargazers)
[](https://github.com/wangyongjie-ntu/Awesome-explainable-AI)
[](https://github.com/wangyongjie-ntu/Awesome-explainable-AI/watchers)
[](https://github.com/wangyongjie-ntu/Awesome-explainable-AI/network/members)
[](https://github.com/wangyongjie-ntu/Awesome-explainable-AI/network/members)
# Awesome-explainable-AI
This repository contains the frontier research on explainable AI(XAI) which is a hot topic recently. From the figure below we can see the trend of interpretable/explainable AI. The publications on this topic are booming.
The figure below illustrates several use cases of XAI. Here we also divide the publications into serveal categories based on this figure. It is challenging to organise these papers well. Good to hear your voice!
## Survey Papers
[The elephant in the interpretability room: Why use attention as explanation when we have saliency methods](https://arxiv.org/abs/2010.05607), EMNLP Workshop 2020
[Explainable Machine Learning in Deployment](https://dl.acm.org/doi/pdf/10.1145/3351095.3375624), FAT 2020
[A brief survey of visualization methods for deep learning models from the perspective of Explainable AI](https://www.macs.hw.ac.uk/~ic14/IoannisChalkiadakis_RRR.pdf), Information Visualization 2020
[Explaining Explanations in AI](https://arxiv.org/pdf/1811.01439.pdf), ACM FAT 2019
[Machine learning interpretability: A survey on methods and metrics](https://www.mdpi.com/2079-9292/8/8/832), Electronics, 2019
[A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI](http://arxiv.org/abs/1907.07374), IEEE TNNLS 2020
[Interpretable machine learning: definitions, methods, and applications](https://arxiv.org/pdf/1901.04592.pdf), Arxiv preprint 2019
[Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers](https://ieeexplore.ieee.org/document/8371286), IEEE Transactions on Visualization and Computer Graphics, 2019
[Explainable Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI](http://arxiv.org/abs/1910.10045), Information Fusion, 2019
[Evaluating Explanation Without Ground Truth in Interpretable Machine Learning](https://arxiv.org/pdf/1907.06831v1.pdf), Arxiv preprint 2019
[A survey of methods for explaining black box models](http://arxiv.org/abs/1802.01933), ACM Computing Surveys, 2018
[Explaining Explanations: An Overview of Interpretability of Machine Learning](https://arxiv.org/abs/1806.00069), IEEE DSAA, 2018
[Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)](https://ieeexplore.ieee.org/document/8466590/), IEEE Access, 2018
[Explainable artificial intelligence: A survey](https://ieeexplore.ieee.org/document/8400040/), MIPRO, 2018
[How Convolutional Neural Networks See the World A Survey of Convolutional Neural Network Visualization Methods](https://arxiv.org/pdf/1804.11191.pdf), Mathematical Foundations of Computing 2018
[Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Models](https://arxiv.org/abs/1708.08296), Arxiv 2017
[Towards A Rigorous Science of Interpretable Machine Learning](https://arxiv.org/pdf/1702.08608.pdf), Arxiv preprint 2017
[Explaining Explanation, Part 1: Theoretical Foundations](https://ieeexplore.ieee.org/abstract/document/7933919), IEEE Intelligent System 2017
[Explaining Explanation, Part 2: Empirical Foundations](https://ieeexplore.ieee.org/abstract/document/8012316), IEEE Intelligent System 2017
[Explaining Explanation, Part 3: The Causal Landscape](https://ieeexplore.ieee.org/abstract/document/8378482), IEEE Intelligent System 2017
[Explaining Explanation, Part 4: A Deep Dive on Deep Nets](https://ieeexplore.ieee.org/abstract/document/8423529), IEEE Intelligent System 2017
[An accurate comparison of methods for quantifying variable importance in artificial neural networks using simulated data](https://depts.washington.edu/oldenlab/wordpress/wp-content/uploads/2013/03/EcologicalModelling_2004.pdf), Ecological Modelling 2004
[Review and comparison of methods to study the contribution of variables in artificial neural network models](http://sovan.lek.free.fr/publi/160-3%20Gevrey.pdf), Ecological Modelling 2003
## Books
[Explainable Artificial Intelligence (xAI) Approaches and Deep Meta-Learning Models](https://www.intechopen.com/online-first/explainable-artificial-intelligence-xai-approaches-and-deep-meta-learning-models), Advances in Deep Learning Chapter 2020
[Explainable AI: Interpreting, Explaining and Visualizing Deep Learning](http://link.springer.com/10.1007/978-3-030-28954-6), Springer 2019
[Explanation in Artificial Intelligence: Insights from the Social Sciences](https://arxiv.org/pdf/1706.07269.pdf), 2017 arxiv preprint
[Visualizations of Deep Neural Networks in Computer Vision: A Survey](https://link.springer.com/chapter/10.1007/978-3-319-54024-5_6), Springer Transparent Data Mining for Big and Small Data 2017
[Explanatory Model Analysis Explore, Explain and Examine Predictive Models](https://pbiecek.github.io/ema/)
[Interpretable Machine Learning A Guide for Making Black Box Models Explainable](https://christophm.github.io/interpretable-ml-book/)
[An Introduction to Machine Learning Interpretability An Applied Perspective on Fairness, Accountability, Transparency,and Explainable AI](https://www.h2o.ai/wp-content/uploads/2019/08/An-Introduction-to-Machine-Learning-Interpretability-Second-Edition.pdf)
## Open Courses
[Interpretability and Explainability in Machine Learning, Harvard University](https://interpretable-ml-class.github.io/)
## Papers
We mainly follow the taxonomy in the [survey paper](http://arxiv.org/abs/1802.01933) and divide the XAI/XML papers into the several branches.
* [1. Transparent Model Design](https://github.com/wangyongjie-ntu/Awesome-explainable-AI/tree/master/transparent_model)
* [2. Post-Explanation](https://github.com/wangyongjie-ntu/Awesome-explainable-AI)
* [2.1 Model Explanation(Model-level)](https://github.com/wangyongjie-ntu/Awesome-explainable-AI/tree/master/model_explanation)
* [2.2 Model Inspection](https://github.com/wangyongjie-ntu/Awesome-explainable-AI/tree/master/model_inspection)
* [2.3 Outcome Explanation](https://github.com/wangyongjie-ntu/Awesome-explainable-AI)
* [2.3.1 Feature Attribution/Importance(Saliency Map)](https://github.com/wangyongjie-ntu/Awesome-explainable-AI/tree/master/feature_attribution)
* [2.4 Neuron Importance](https://github.com/wangyongjie-ntu/Awesome-explainable-AI/tree/master/neuron_importance)
* [2.5 Example-based Explanations](https://github.com/wangyongjie-ntu/Awesome-explainable-AI)
* [2.5.1 Counterfactual Explanations(Recourse)](https://github.com/wangyongjie-ntu/Awesome-explainable-AI/tree/master/counterfactuals)
* [2.5.2 Influential Instances](https://github.com/wangyongjie-ntu/Awesome-explainable-AI/tree/master/influential_instances)
* [2.5.3 Prototypes&Criticisms](https://github.com/wangyongjie-ntu/Awesome-explainable-AI/tree/master/prototype_criticisms)
### Uncategorized Papers on Model/Instance Explanation
[Incorporating Interpretable Output Constraints in Bayesian Neural Networks](https://arxiv.org/abs/2010.10969), NeuIPS 2020
[Towards Interpretable Natural Language Understanding with Explanations as Latent Variables](https://arxiv.org/abs/2011.05268), NeurIPS 2020
[Learning identifiable and interpretable latent models of high-dimensional neural activity using pi-VAE](https://arxiv.org/abs/2011.04798), NeurIPS 2020
[Generative causal explanations of black-box classifiers](https://arxiv.org/abs/2006.13913), NeurIPS 2020
[Learning outside the Black-Box: The pursuit of interpretable models](https://proceedings.neurips.cc//paper/2020/file/ce758408f6ef98d7c7a7b786eca7b3a8-Paper.pdf), NeurIPS 2020
[Explaining Groups of Points in Low-Dimensional Representations](https://arxiv.org/pdf/2003.01640.pdf), ICML 2020
[Explaining Knowledge Distillation by Quantifying the Knowledge](https://openaccess.thecvf.com/content_CVPR_2020/papers/Cheng_Explaining_Knowledge_Distillation_by_Quantifying_the_Knowledge_CVPR_2020_paper.pdf), CVPR 2020
[Fanoos: Multi-Resolution, Multi-Strength, Interactive Explanations for Learned Systems](https://arxiv.org/pdf/2006.12453.pdf), IJCAI 2020
[Machine Learning Explainability for External Stakeholders](https://arxiv.org/abs/2007.05408), IJCAI 2020
[Py-CIU: A Python Library for Explaining Machine Learning Predictions Using Contextual Importance and Utility](https://www.researchgate.net/publication/344154017_Py-CIU_A_Python_Library_for_Explaining_Machine_Learning_Predictions_Using_Contextual_Importance_and_Utility), IJCAI 2020
[Machine Learning Explainability for External Stakeholders](https://arxiv.org/abs/2007.05408), IJCAI 2020
[Interpretable Models for Understanding Immersive Simulations](https://www.ijcai.org/Proceedings/2020/0321.pdf), IJCAI 2020
[Towards Automatic Concept-based Explanations](https://arxiv.org/pdf/1902.03129.pdf), NIPS 2019
[Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead](https://arxiv.org/pdf/1811.10154.pdf), Nature Machine Intelligence 2019
[Interpretml: A unified framework for machine learning interpretability](https://arxiv.org/abs/1909.09223), arxiv preprint 2019
[All Models are Wrong, but Many are Useful: Learning a Variables Importance by Studying an Entire Class of Prediction Models Simultaneously](https://arxiv.org/pdf/1801.01489.pdf), JMLR 2019
[On the Robustness of Interpretability Methods](https://arxiv.org/abs/1806.08049), ICML 2018 workshop
[Towards A Rigorous Science of Interpretable Machine Learning](https://arxiv.org/pdf/1702.08608.pdf), Arxiv preprint 2017
[Object Region Mining With Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach](https://openaccess.thecvf.com/content_cvpr_2017/papers/Wei_Object_Region_Mining_CVPR_2017_paper.pdf), CVPR 2017
LOCO, [Distribution-Free Predictive Inference For Regression](https://arxiv.org/pdf/1604.04173.pdf), Arxiv preprint 2016
[Explaining data-driven document classifications](https://www.jstor.org/stable/26554869), MIS Quarterly 2014
### Evaluation methods
[Evaluations and Methods for Explanation through Robustness Analysis](https://arxiv.org/pdf/2006.00442.pdf), arxiv preprint 2020
[Evaluating and Aggregating Feature-based Model Explanations](https://arxiv.org/abs/2005.00631), IJCAI 2020
[Sanity Checks for Saliency Metrics](https://aaai.org/ojs/index.php/AAAI/article/view/6064), AAAI 2020
[A benchmark for interpretability methods in deep neural networks](https://papers.nips.cc/paper/9167-a-benchmark-for-interpretability-methods-in-deep-neural-networks.pdf), NIPS 2019
[Methods for interpreting and understanding deep neural networks](https://www.sciencedirect.com/science/article/pii/S1051200417302385), Digital Signal Processing 2017
[Evaluating the visualization of what a Deep Neural Network has learned](http://arxiv.org/abs/1509.06321), IEEE Transactions on Neural Networks and Learning Systems 2015
## Python Libraries(sort in alphabeta order)
AIF360: [https://github.com/Trusted-AI/AIF360](https://github.com/Trusted-AI/AIF360), 
AIX360: [https://github.com/IBM/AIX360](https://github.com/IBM/AIX360), 
Anchor: [https://github.com/marcotcr/anchor](https://github.com/marcotcr/anchor), scikit-learn 
Alibi: [https://github.com/SeldonIO/alibi](https://github.com/SeldonIO/alibi) 
Alibi-detect: [https://github.com/SeldonIO/alibi-detect](https://github.com/SeldonIO/alibi-detect) 
BlackBoxAuditing: [https://github.com/algofairness/BlackBoxAuditing](https://github.com/algofairness/BlackBoxAuditing), scikit-learn 
Boruta-Shap: [https://github.com/Ekeany/Boruta-Shap](https://github.com/Ekeany/Boruta-Shap), scikit-learn 
casme: [https://github.com/kondiz/casme](https://github.com/kondiz/casme), Pytorch 
Captum: [https://github.com/pytorch/captum](https://github.com/pytorch/captum), Pytorch, 
cnn-exposed: [https://github.com/idealo/cnn-exposed](https://github.com/idealo/cnn-exposed), Tensorflow 
DALEX: [https://github.com/ModelOriented/DALEX](https://github.com/ModelOriented/DALEX), 
Deeplift: [https://github.com/kundajelab/deeplift](https://github.com/kundajelab/deeplift), Tensorflow, Keras
DeepExplain: [https://github.com/marcoancona/DeepExplain](https://github.com/marcoancona/DeepExplain), Tensorflow, Keras 
Deep Visualization Toolbox: [https://github.com/yosinski/deep-visualization-toolbox](https://github.com/yosinski/deep-visualization-toolbox), Caffe, 
Eli5: [https://github.com/TeamHG-Memex/eli5](https://github.com/TeamHG-Memex/eli5), Scikit-learn, Keras, xgboost, lightGBM, catboost etc.
explainx: [https://github.com/explainX/explainx](https://github.com/explainX/explainx), xgboost, catboost 
Grad-cam-Tensorflow: [https://github.com/insikk/Grad-CAM-tensorflow](https://github.com/insikk/Grad-CAM-tensorflow), Tensorflow 
Innvestigate: [https://github.com/albermax/innvestigate](https://github.com/albermax/innvestigate), tensorflow, theano, cntk, Keras 
imodels: [https://github.com/csinva/imodels](https://github.com/csinva/imodels), 
InterpretML: [https://github.com/interpretml/interpret](https://github.com/interpretml/interpret) 
interpret-community: [https://github.com/interpretml/interpret-community](https://github.com/interpretml/interpret-community) 
Integrated-Gradients: [https://github.com/ankurtaly/Integrated-Gradients](https://github.com/ankurtaly/Integrated-Gradients), Tensorflow 
Keras-grad-cam: [https://github.com/jacobgil/keras-grad-cam](https://github.com/jacobgil/keras-grad-cam), Keras 
Keras-vis: [https://github.com/raghakot/keras-vis](https://github.com/raghakot/keras-vis), Keras 
keract: [https://github.com/philipperemy/keract](https://github.com/philipperemy/keract), Keras 
Lucid: [https://github.com/tensorflow/lucid](https://github.com/tensorflow/lucid), Tensorflow 
LIT: [https://github.com/PAIR-code/lit](https://github.com/PAIR-code/lit), Tensorflow, specified for NLP Task 
Lime: [https://github.com/marcotcr/lime](https://github.com/marcotcr/lime), Nearly all platform on Python 
LOFO: [https://github.com/aerdem4/lofo-importance](https://github.com/aerdem4/lofo-importance), scikit-learn 
modelStudio: [https://github.com/ModelOriented/modelStudio](https://github.com/ModelOriented/modelStudio), Keras, Tensorflow, xgboost, lightgbm, h2o 
pytorch-cnn-visualizations: [https://github.com/utkuozbulak/pytorch-cnn-visualizations](https://github.com/utkuozbulak/pytorch-cnn-visualizations), Pytorch 
Pytorch-grad-cam: [https://github.com/jacobgil/pytorch-grad-cam](https://github.com/jacobgil/pytorch-grad-cam), Pytorch 
PDPbox: [https://github.com/SauceCat/PDPbox](https://github.com/SauceCat/PDPbox), Scikit-learn 
py-ciu:[https://github.com/TimKam/py-ciu/](https://github.com/TimKam/py-ciu/), 
PyCEbox: [https://github.com/AustinRochford/PyCEbox](https://github.com/AustinRochford/PyCEbox) 
path_explain: [https://github.com/suinleelab/path_explain](https://github.com/suinleelab/path_explain), Tensorflow 
rulefit: [https://github.com/christophM/rulefit](https://github.com/christophM/rulefit), 
rulematrix: [https://github.com/rulematrix/rule-matrix-py](https://github.com/rulematrix/rule-matrix-py), 
Saliency: [https://github.com/PAIR-code/saliency](https://github.com/PAIR-code/saliency), Tensorflow 
SHAP: [https://github.com/slundberg/shap](https://github.com/slundberg/shap), Nearly all platform on Python 
Skater: [https://github.com/oracle/Skater](https://github.com/oracle/Skater) 
TCAV: [https://github.com/tensorflow/tcav](https://github.com/tensorflow/tcav), Tensorflow, scikit-learn 
skope-rules: [https://github.com/scikit-learn-contrib/skope-rules](https://github.com/scikit-learn-contrib/skope-rules), Scikit-learn 
TensorWatch: [https://github.com/microsoft/tensorwatch.git](https://github.com/microsoft/tensorwatch.git), Tensorflow 
tf-explain: [https://github.com/sicara/tf-explain](https://github.com/sicara/tf-explain), Tensorflow 
Treeinterpreter: [https://github.com/andosa/treeinterpreter](https://github.com/andosa/treeinterpreter), scikit-learn, 
WeightWatcher: [https://github.com/CalculatedContent/WeightWatcher](https://github.com/CalculatedContent/WeightWatcher), Keras, Pytorch 
What-if-tool: [https://github.com/PAIR-code/what-if-tool](https://github.com/PAIR-code/what-if-tool), Tensorflow
XAI: [https://github.com/EthicalML/xai](https://github.com/EthicalML/xai), scikit-learn 
## Related Repositories
[https://github.com/jphall663/awesome-machine-learning-interpretability](https://github.com/jphall663/awesome-machine-learning-interpretability), 
[https://github.com/lopusz/awesome-interpretable-machine-learning](https://github.com/lopusz/awesome-interpretable-machine-learning), 
[https://github.com/pbiecek/xai_resources](https://github.com/pbiecek/xai_resources), 
[https://github.com/h2oai/mli-resources](https://github.com/h2oai/mli-resources), 
## Acknowledge
Need your help to re-organize and refine current taxonomy. Thanks very very much!
I appreciate it very much if you could add more works related to XAI/XML to this repo, archive uncategoried papers or anything to enrich this repo.
If any questions, feel free to contact me(yongjie.wang@ntu.edu.sg) or discuss on [Gitter **Chat**](https://gitter.im/explainable_ai/community). Welcome to discuss together.
Owner
- Name: André Artelt
- Login: andreArtelt
- Kind: user
- Location: Germany
- Company: Bielefeld University
- Repositories: 3
- Profile: https://github.com/andreArtelt
PhD student