https://github.com/amazon-science/manifold-restricted-shapley

Science Score: 36.0%

This score indicates how likely this project is to be science-related based on various indicators:

○
CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
○
DOI references
✓
Academic publication links
Links to: arxiv.org, scholar.google
○
Academic email domains
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (8.5%) to scientific vocabulary

Last synced: 9 months ago · JSON representation

Repository

Basic Info

Host: GitHub
Owner: amazon-science
License: apache-2.0
Language: Jupyter Notebook
Default Branch: main
Size: 4.5 MB

Statistics

Stars: 5
Watchers: 3
Forks: 2
Open Issues: 1
Releases: 0

Archived

Created over 3 years ago · Last pushed almost 3 years ago

Metadata Files

Readme Contributing License Code of conduct

Manifold Restricted Interventional Shapley Values

By Muhammad Faaiz Taufiq, Patrick Blöbaum, Lenon Minorics.

This project is the official implementation of "Manifold Restricted Interventional Shapley Values".

Introduction

Shapley values are model-agnostic methods for explaining model predictions. Many commonly used methods of computing Shapley values, known as off-manifold methods, rely on model evaluations on out-of-distribution input samples. Consequently, explanations obtained are sensitive to model behaviour outside the data distribution, which may be irrelevant for all practical purposes. While on-manifold methods have been proposed which do not suffer from this problem, we show that such methods are overly dependent on the input data distribution, and therefore result in unintuitive and misleading explanations. To circumvent these problems, we propose ManifoldShap, which respects the model’s domain of validity by restricting model evaluations to the data manifold. We show, theoretically and empirically, that ManifoldShap is robust to off-manifold perturbations of the model and leads to more accurate and intuitive explanations than existing state-of-the-art Shapley methods.

teaser

Installation

To install this package, run the following command:
pip install -r requirements.txt

Running Synthetic Data Experiments

Here, we outline the commands to run the synthetic data experiments. For each experiment, the parameters can be changed using the arguments provided in the arguments.py file.

Off-manifold perturbation (regression) example

To run the experiments with the perturbed model $g\delta$ defined as:
$g\delta(x1, x2) \coloneqq x1 + \delta x2 1((x1,x2) \notin \mathcal{P}\alpha)$
run the command:
```python perturbationexperimentwithdifferent_dag.py```

Off-manifold perturbation (classification) example

To instead run the experiments with perturbed classifier model $g\delta$ defined as:
$g\delta(x1, x2) \coloneqq 1(x1 > 1/2, (x1, x2)\in \mathcal{P}\alpha) + 1(x1 (1-\delta) > 1/2, (x1,x2) \notin \mathcal{P}\alpha)$
run the command:
python perturbation_off_manifold_main.py

Increasing correlations example

To run the experiment with increasing feature correlations, run the command:
python shap_values_for_increasing_correlation.py

Varying manifold size experiments

To run the experiment with varying manifold sizes, run the command:
python shap_values_vs_epsilon_values.py

ManifoldShap vs RJBShap experiments

To run the experiment comparing ManifoldShap with RJBShap, run the command:
python density_exp.py

Running Real-world Dataset Experiments

To run the real-world dataset experiments, please refer to the detailed steps provided in fooling-shap-experiments.

Security

See CONTRIBUTING for more information.

License

This project is licensed under the Apache-2.0 License.

Owner

Name: Amazon Science
Login: amazon-science
Kind: organization

Website: https://amazon.science
Twitter: AmazonScience
Repositories: 80
Profile: https://github.com/amazon-science

GitHub Events

Total

Last Year

Issues and Pull Requests

Last synced: over 1 year ago

All Time

Total issues: 0
Total pull requests: 6
Average time to close issues: N/A
Average time to close pull requests: about 4 hours
Total issue authors: 0
Total pull request authors: 2
Average comments per issue: 0
Average comments per pull request: 0.5
Merged pull requests: 2
Bot issues: 0
Bot pull requests: 4

Past Year

Issues: 0
Pull requests: 0
Average time to close issues: N/A
Average time to close pull requests: N/A
Issue authors: 0
Pull request authors: 0
Average comments per issue: 0
Average comments per pull request: 0
Merged pull requests: 0
Bot issues: 0
Bot pull requests: 0

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Open Source Science