Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/holistic-ai/holisticai

This is an open-source tool to assess and improve the trustworthiness of AI systems.

https://github.com/holistic-ai/holisticai

Science Score: 26.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (13.5%) to scientific vocabulary

Keywords

benchmark bias-mitigation efficacy explainability-metric explainable-ai machine-learning privacy responsible-ai robustness
Last synced: 5 months ago · JSON representation

Repository

This is an open-source tool to assess and improve the trustworthiness of AI systems.

Basic Info
Statistics
  • Stars: 90
  • Watchers: 2
  • Forks: 24
  • Open Issues: 16
  • Releases: 15
Topics
benchmark bias-mitigation efficacy explainability-metric explainable-ai machine-learning privacy responsible-ai robustness
Created about 3 years ago · Last pushed 10 months ago
Metadata Files
Readme Changelog Contributing License Codeowners

README.md


Holistic AI: building trustworthy AI systems

PyPI Documentation Status PyPI - License PyPI - Downloads Slack

Holistic AI is an open-source library dedicated to assessing and improving the trustworthiness of AI systems. We believe that responsible AI development requires a comprehensive evaluation across multiple dimensions, beyond just accuracy.

Current Capabilities

Holistic AI currently focuses on five verticals of AI trustworthiness:

  1. Bias: measure and mitigate bias in AI models.
  2. Explainability: measure into model behavior and decision-making.
  3. Robustness: measure model performance under various conditions.
  4. Security: measure the privacy risks associated with AI models.
  5. Efficacy: measure the effectiveness of AI models.

Quick Start

bash pip install holisticai # Basic installation pip install holisticai[datasets] # add datasets and plot dependencies pip install holisticai[bias] # Bias mitigation support pip install holisticai[explainability] # For explainability metrics and plots pip install holisticai[all] # Install all packages for bias and explainability

```python

imports

from holisticai.bias.metrics import classificationbiasmetrics from holisticai.datasets import loaddataset from holisticai.bias.plots import biasmetricsreport from sklearn.linearmodel import LogisticRegression from sklearn.preprocessing import StandardScaler

load an example dataset and split

dataset = loaddataset('lawschool', protectedattribute="race") datasetsplit = dataset.traintestsplit(test_size=0.3)

separate the data into train and test sets

traindata = datasetsplit['train'] testdata = datasetsplit['test']

rescale the data

scaler = StandardScaler() Xtraint = scaler.fittransform(traindata['X']) Xtestt = scaler.transform(test_data['X'])

train a logistic regression model

model = LogisticRegression(randomstate=42, maxiter=500) model.fit(Xtraint, train_data['y'])

make predictions

ypred = model.predict(Xtest_t)

compute bias metrics

metrics = classificationbiasmetrics( groupa = testdata['groupa'], groupb = testdata['groupb'], ytrue = testdata['y'], ypred = ypred )

create a comprehensive report

biasmetricsreport(modeltype='binaryclassification', table_metrics=metrics) ```

Key Features

  • Comprehensive Metrics: Measure various aspects of AI system trustworthiness, including bias, fairness, and explainability.
  • Mitigation Techniques: Implement strategies to address identified issues and improve the fairness and robustness of AI models.
  • User-Friendly Interface: Intuitive API for easy integration into existing workflows.
  • Visualization Tools: Generate insightful visualizations for better understanding of model behavior and bias patterns.

Documentation and Tutorials

Detailed Installation

Troubleshooting (macOS):

Before installing the library, you may need to install these packages:

bash brew install cbc pkg-config python -m pip install cylp brew install cmake

Contributing

We welcome contributions from the community To learn more about contributing to Holistic AI, please refer to our Contributing Guide.

Owner

  • Name: Holistic AI
  • Login: holistic-ai
  • Kind: organization
  • Email: we@holisticai.com

GitHub Events

Total
  • Create event: 52
  • Release event: 9
  • Issues event: 11
  • Watch event: 22
  • Delete event: 35
  • Issue comment event: 11
  • Push event: 120
  • Pull request review event: 19
  • Pull request review comment event: 5
  • Pull request event: 95
  • Fork event: 11
Last Year
  • Create event: 52
  • Release event: 9
  • Issues event: 11
  • Watch event: 22
  • Delete event: 35
  • Issue comment event: 11
  • Push event: 120
  • Pull request review event: 19
  • Pull request review comment event: 5
  • Pull request event: 95
  • Fork event: 11

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 0
  • Total pull requests: 15
  • Average time to close issues: N/A
  • Average time to close pull requests: 19 days
  • Total issue authors: 0
  • Total pull request authors: 3
  • Average comments per issue: 0
  • Average comments per pull request: 0.07
  • Merged pull requests: 4
  • Bot issues: 0
  • Bot pull requests: 8
Past Year
  • Issues: 0
  • Pull requests: 15
  • Average time to close issues: N/A
  • Average time to close pull requests: 19 days
  • Issue authors: 0
  • Pull request authors: 3
  • Average comments per issue: 0
  • Average comments per pull request: 0.07
  • Merged pull requests: 4
  • Bot issues: 0
  • Bot pull requests: 8
View more stats
Top Authors
Issue Authors
  • crismunoz (27)
  • Kleyt0n (3)
  • fracarfer5 (2)
  • neoleeax (1)
  • dependabot[bot] (1)
  • andrelfnovaes (1)
Pull Request Authors
  • crismunoz (71)
  • Kleyt0n (25)
  • fracarfer5 (24)
  • dependabot[bot] (24)
  • aminatkhamokova (7)
  • andrelfnovaes (5)
  • 981526092 (3)
  • neoleeax (2)
Top Labels
Issue Labels
enhancement (2) documentation (1) bias (1) hackaton (1) dependencies (1) python (1)
Pull Request Labels
dependencies (25) github_actions (12) python (10) documentation (2)

Packages

  • Total packages: 2
  • Total downloads:
    • pypi 11,919 last-month
  • Total dependent packages: 0
    (may contain duplicates)
  • Total dependent repositories: 1
    (may contain duplicates)
  • Total versions: 28
  • Total maintainers: 4
pypi.org: holisticai
  • Versions: 27
  • Dependent Packages: 0
  • Dependent Repositories: 1
  • Downloads: 11,833 Last month
Rankings
Downloads: 9.4%
Dependent packages count: 10.0%
Average: 13.7%
Dependent repos count: 21.7%
Maintainers (4)
GBadila holisticgiuliofilippi sarazannone crismunoz
Last synced: 6 months ago
pypi.org: holistic
  • Versions: 1
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Downloads: 86 Last month
Rankings
Dependent packages count: 10.1%
Average: 33.4%
Dependent repos count: 56.6%
Maintainers (1)
crismunoz
Last synced: about 1 year ago

Dependencies

.github/workflows/pr-workflow.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
  • pre-commit/action v3.0.0 composite
.github/workflows/release-workflow.yml actions
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
docs/requirements.txt pypi
  • Sphinx *
  • holisticai *
  • myst_parser *
  • nbsphinx *
  • sphinx-copybutton *
  • sphinx-rtd-theme *
  • sphinxcontrib-napoleon *
poetry.lock pypi
  • argcomplete 2.0.0 develop
  • attrs 22.2.0 develop
  • cfgv 3.3.1 develop
  • charset-normalizer 2.1.1 develop
  • commitizen 2.40.0 develop
  • decli 0.5.2 develop
  • distlib 0.3.6 develop
  • exceptiongroup 1.1.0 develop
  • filelock 3.9.0 develop
  • flake8 5.0.4 develop
  • identify 2.5.15 develop
  • iniconfig 2.0.0 develop
  • jinja2 3.1.2 develop
  • markupsafe 2.1.2 develop
  • mccabe 0.7.0 develop
  • nodeenv 1.7.0 develop
  • platformdirs 2.6.2 develop
  • pluggy 1.0.0 develop
  • pre-commit 2.21.0 develop
  • prompt-toolkit 3.0.36 develop
  • pycodestyle 2.9.1 develop
  • pyflakes 2.5.0 develop
  • pytest 7.2.1 develop
  • pyyaml 6.0 develop
  • questionary 1.10.0 develop
  • termcolor 2.2.0 develop
  • tomli 2.0.1 develop
  • tomlkit 0.11.6 develop
  • typing-extensions 4.4.0 develop
  • virtualenv 20.16.2 develop
  • wcwidth 0.2.6 develop
  • colorama 0.4.6
  • cvxopt 1.3.0
  • cvxpy 1.3.0
  • cycler 0.11.0
  • cylp 0.91.5
  • ecos 2.0.12
  • fonttools 4.38.0
  • joblib 1.2.0
  • kiwisolver 1.4.4
  • matplotlib 3.5.3
  • networkx 3.0
  • numpy 1.21.6
  • osqp 0.6.2.post8
  • packaging 23.0
  • pandas 1.1.5
  • pillow 9.4.0
  • pyparsing 3.0.9
  • python-dateutil 2.8.2
  • pytz 2022.7.1
  • qdldl 0.1.5.post3
  • scikit-learn 1.0.2
  • scipy 1.7.3
  • scs 3.2.2
  • seaborn 0.12.2
  • setuptools 64.0.2
  • six 1.16.0
  • threadpoolctl 3.1.0
  • tqdm 4.64.1
pyproject.toml pypi
  • cvxopt ^1.3.0
  • cvxpy ^1.3.0
  • python >=3.8,<3.11
  • scikit-learn >=1.0.2
  • seaborn >=0.11.2
  • tqdm ^4.64.1