Introducing DIANNA video

Deep Insight And Neural Network Analysis

DIANNA is a Python package that brings explainable AI (XAI) to your research project. It wraps carefully selected XAI methods in a simple, uniform interface. It's built by, with and for (academic) researchers and research software engineers working on machine learning projects.

Why DIANNA?

DIANNA software is addressing needs of both (X)AI researchers and mostly the various domains scientists who are using or will use AI models for their research without being experts in (X)AI. DIANNA is future-proof: one of the very few XAI library supporting the Open Neural Network Exchange (ONNX) format.

After studying the vast XAI landscape we have made choices in the parts of the XAI Taxonomy on which methods, data modalities and problems types to focus. Our choices, based on the largest usage in scientific literature, are shown graphically in the XAI taxonomy below:

The key points of DIANNA:

• Provides an easy-to-use interface for non (X)AI experts
• Implements well-known XAI methods LIME, RISE and KernelSHAP, chosen by systematic and objective evaluation criteria
• Comes with a dashboard where results of different explainers can be compared for all data types
• Supports the de-facto standard of neural network models - ONNX
• Supports images, text, time series, tabular data modalities and embeddings (in a related package)
• Comes with simple intuitive image, text, time series, and tabular benchmarks, so can help you with your XAI research
• Includes scientific use-cases tutorials
• Easily extendable to other XAI methods

For more information on the unique strengths of DIANNA with comparison to other tools, please see the context landscape.

Installation

DIANNA can be installed from PyPI using pip on any of the supported Python versions (see badge):

console python3 -m pip install dianna

To install the most recent development version directly from the GitHub repository run:

console python3 -m pip install git+https://github.com/dianna-ai/dianna.git

If you get an error related to OpenMP when importing dianna, have a look at this issue for possible workarounds.

Getting started

You need:

• your trained ONNX model (convert my pytorch/tensorflow/keras/scikit-learn model to ONNX)
• a data item to be explained

You get:

• a relevance map overlayed over the data item

Template example for any data modality and explainer

1. Provide your trained model and data item ( text, image, time series or tabular )

python model_path = 'your_model.onnx' # model trained on your data modality data_item = <data_item> # data item for which the model's prediction needs to be explained

1. If the task is classification: which are the classes your model has been trained for?

python labels = [class_a, class_b] # example of binary classification labels Which of these classes do you want an explanation for? python explained_class_index = labels.index(<explained_class>) # explained_class can be any of the labels

1. Run dianna with the explainer of your choice ( 'LIME', 'RISE' or 'KernalSHAP') and visualize the output:

python explanation = dianna.<explanation_function>(model_path, data_item, explainer) dianna.visualization.<visualization_function>(explanation[explained_class_index], data_item)

Text and image usage

Overview tutorial

There are full working examples on how to use the supported explainers and how to use dianna for all supported data modalities in our overview tutorial.

IMPORTANT: Sensitivity to hyperparameters

The explainers are sensitive to the choice of their hyperparameters! In this work, this sensitivity to hyperparameters is researched and useful conclusions are drawn. The default hyperparameters used in DIANNA for each explainer as well as the values for our tutorial examples are given in the Tutorials README.

Introductory video

This video shows the main functionality of DIANNA and shows you how to use DIANNA also from its dashboard.

Watch the video on YouTube

Dashboard

Explore the explanations of your trained model using the DIANNA dashboard. Click here for more information.

Datasets

DIANNA comes with simple datasets. Their main goal is to provide intuitive insight into the working of the XAI methods. They can be used as benchmarks for evaluation and comparison of existing and new XAI methods.

Images

| Dataset | Description | Examples | Generation | | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------- | | Binary MNIST | Greyscale images of the digits "1" and "0" - a 2-class subset from the famous[MNIST dataset](http://yann.lecun.com/exdb/mnist/) for handwritten digit classification. | | [Binary MNIST dataset generation](https://github.com/dianna-ai/dianna-exploration/tree/main/example_data/dataset_preparation/MNIST) | | [Simple Geometric (circles and triangles)](https://doi.org/10.5281/zenodo.5012824) | Images of circles and triangles for 2-class geometric shape classificaiton. The shapes of varying size and orientation and the background have varying uniform gray levels. | | [Simple geometric shapes dataset generation](https://github.com/dianna-ai/dianna-exploration/tree/main/example_data/dataset_preparation/geometric_shapes) | | [Simple Scientific (LeafSnap30)](https://zenodo.org/record/5061353/) | Color images of tree leaves - a 30-class post-processed subset from the LeafSnap dataset for automatic identification of North American tree species. | | [LeafSnap30 dataset generation](https://github.com/dianna-ai/dianna-exploration/blob/main/example_data/dataset_preparation/LeafSnap/) |

Models

We work with ONNX! ONNX is a great unified neural network standard which can be used to boost reproducible science. Using ONNX for your model also gives you a boost in performance! In case your models are still in another popular DNN (deep neural network) format, here are some simple recipes to convert them:

• pytorch and pytorch-lightning - use the built-in torch.onnx.export function to convert pytorch models to onnx, or call the built-in to_onnx function on your LightningModule to export pytorch-lightning models to onnx.
• tensorflow - use the tf2onnx package to convert tensorflow models to onnx.
• keras - same as the conversion from tensorflow to onnx, the tf2onnx package also supports keras.
• scikit-learn - use the skl2onnx package to scikit-learn models to onnx.

More converters with examples and tutorials can be found on the ONNX tutorial page.

And here are links to notebooks showing how we created our models on the benchmark datasets:

We envision the birth of the ONNX Scientific models zoo soon...

Tutorials

DIANNA supports different data modalities and XAI methods (explainers). We have evaluated many explainers using objective criteria (see the How to find your AI explainer blog-post). The table below contains links to the relevant XAI method's papers (for some explanatory videos on the methods, please see tutorials). The DIANNA tutorials cover each supported method and data modality on a least one dataset using the default or tuned hyperparameters. Our plans to expand DIANNA with more data modalities and explainers are given in the ROADMAP.

| Data \ XAI | RISE | LIME | KernelSHAP | | :--------- | :------------------------------------------------ | :----------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------- | | Images | ✅ | ✅ | ✅ | | Text | ✅ | ✅ | | | Timeseries | ✅ | ✅ | | | Tabular | ✅ | ✅ | ✅ | | Embedding | inspired by RISE in distance_explainer | | | | Graphs | next steps | ... | ... |

LRP and PatternAttribution also feature in the top 5 of our thoroughly evaluated explainers. Also GradCAM) has been recently found to be semantically continous! Contributing by adding these and more (new) post-hoc explainability methods on ONNX models is very welcome!

Scientific use-cases

Our goal is that the scientific community embrases XAI as a source for novel and unexplored perspectives on scientific problems. Here, we offer tutorials on specific scientific use-cases of uisng XAI:

| Use-case (data) \ XAI | RISE | LIME | KernelSHAP | |:-------------------------------------------------------------------|:-----------------------------------------------------| :---------------------------------------------------------------------| :-------------------------------------------------------------------------------------------------------| | Biology (Phytomorphology): Tree Leaves classification (images) | | ✅ | | | Astronomy: Fast Radio Burst detection (timeseries) | ✅ | | | | Land-atmosphere modeling: Latent heat flux prediction (tabular) | | | ✅ | | Social sciences: EU-law regulatory statement classification (text) | | ✅ | | | Climate | planned | ... | ... |

Reference documentation

For detailed information on using specific DIANNA functions, please visit the documentation page hosted at Readthedocs.

Contributing

If you want to contribute to the development of DIANNA, have a look at the contribution guidelines. See our developer documentation for information on developer installation, running tests, generating documentation, versioning and making a release.

How to cite us

If you use this package for your scientific work, please consider citing directly the software as:

Ranguelova, E., Bos, P., Liu, Y., Meijer, C., Oostrum, L., Crocioni, G., Ootes, L., Chandramouli, P., Jansen, A., Smeets, S. (2023). dianna (*[VERSION YOU USED]*). Zenodo. https://zenodo.org/record/5592606

or the JOSS paper as:

Ranguelova et al., (2022). DIANNA: Deep Insight And Neural Network Analysis. Journal of Open Source Software, 7(80), 4493, https://doi.org/10.21105/joss.04493

See also the Zenodo page or the JOSS page for exporting the software citation to BibTteX and other formats.

Credits

This package was created with Cookiecutter and the NLeSC/python-template.