fairlearn

A Python package to assess and improve fairness of machine learning models.

Science Score: 54.0%

This score indicates how likely this project is to be science-related based on various indicators:

✓
CITATION.cff file
Found CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
○
DOI references
○
Academic publication links
✓
Committers with academic emails
4 of 104 committers (3.8%) from academic institutions
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (12.2%) to scientific vocabulary

Keywords

ai ai-systems artificial-intelligence fairness fairness-ai fairness-assessment fairness-ml group-fairness harms machine-learning responsible-ai unfairness-mitigation

Keywords from Contributors

developer-tools deep-neural-networks closember distribution embedded interactive caching analyses transformers notebook

Last synced: 6 months ago · JSON representation ·

Repository

A Python package to assess and improve fairness of machine learning models.

Basic Info

Host: GitHub
Owner: fairlearn
License: mit
Language: Python
Default Branch: main
Homepage: https://fairlearn.org
Size: 76.3 MB

Statistics

Stars: 2,110
Watchers: 36
Forks: 464
Open Issues: 135
Releases: 16

Topics

ai ai-systems artificial-intelligence fairness fairness-ai fairness-assessment fairness-ml group-fairness harms machine-learning responsible-ai unfairness-mitigation

Created almost 8 years ago · Last pushed 7 months ago

Metadata Files

Readme License Citation

README.rst

|MIT license| |PyPI| |Discord| |StackOverflow|

Fairlearn
=========

Fairlearn is a Python package that empowers developers of artificial
intelligence (AI) systems to assess their system's fairness and mitigate
any observed unfairness issues. Fairlearn contains mitigation algorithms
as well as metrics for model assessment. Besides the source code, this
repository also contains Jupyter notebooks with examples of Fairlearn
usage.

Website: https://fairlearn.org/

-  `What we mean by *fairness* <#what-we-mean-by-fairness>`__
-  `Overview of Fairlearn <#overview-of-fairlearn>`__
-  `Getting started with fairlearn <#getting-started-with-fairlearn>`__
-  `Maintainers <#maintainers>`__
-  `Code of conduct <#code-of-conduct>`__
-  `Issues <#issues>`__

What we mean by *fairness*
--------------------------

An AI system can behave unfairly for a variety of reasons. In Fairlearn,
we define whether an AI system is behaving unfairly in terms of its
impact on people – i.e., in terms of harms. We focus on two kinds of
harms:

-  *Allocation harms.* These harms can occur when AI systems extend or
   withhold opportunities, resources, or information. Some of the key
   applications are in hiring, school admissions, and lending.

-  *Quality-of-service harms.* Quality of service refers to whether a
   system works as well for one person as it does for another, even if
   no opportunities, resources, or information are extended or withheld.

We follow the approach known as **group fairness**, which asks: *Which
groups of individuals are at risk for experiencing harms?* The relevant
groups need to be specified by the data scientist and are application
specific.

Group fairness is formalized by a set of constraints, which require that
some aspect (or aspects) of the AI system's behavior be comparable
across the groups. The Fairlearn package enables assessment and
mitigation of unfairness under several common definitions. To learn more
about our definitions of fairness, please visit our
`user guide on Fairness of AI Systems `__.

    *Note*: Fairness is fundamentally a sociotechnical challenge. Many
    aspects of fairness, such as justice and due process, are not
    captured by quantitative fairness metrics. Furthermore, there are
    many quantitative fairness metrics which cannot all be satisfied
    simultaneously. Our goal is to enable humans to assess different
    mitigation strategies and then make trade-offs appropriate to their
    scenario.

Overview of Fairlearn
---------------------

The Fairlearn Python package has two components:

-  *Metrics* for assessing which groups are negatively impacted by a
   model, and for comparing multiple models in terms of various fairness
   and accuracy metrics.

-  *Algorithms* for mitigating unfairness in a variety of AI tasks and
   along a variety of fairness definitions.

Fairlearn metrics
~~~~~~~~~~~~~~~~

Check out our in-depth `guide on the Fairlearn metrics `__.

Fairlearn algorithms
~~~~~~~~~~~~~~~~~~~

For an overview of our algorithms please refer to our
`website `__.

Getting Started with Fairlearn
-----------------------------

First steps
~~~~~~~~~~~

- Install via pip: ``pip install fairlearn``

- Visit the `Quickstart guide `__.

- **Learning Resources**:

  - Read the comprehensive `user guide `__.

  - Look through the `example notebooks `__.


For Users & Practitioners
~~~~~~~~~~~~~~~~~~~~~~~~

- Browse the `example gallery `__. 
Please note that notebooks downloaded from `main` may not be compatible with pip-installed versions.

- Check the `API reference `__.

- **Get Help**:

  - Ask questions on `Stack Overflow `__ with tag ``fairlearn``.

  - Join the `Discord community `__ for discussions.

For Contributors
~~~~~~~~~~~~~~~

- Read the `contributor guide `__.

- Check out the `good first issues `__.

- Follow the `development process `__.

- Join the `Discord `__ for contributor discussions. Please use the ``#development`` channel.


Maintainers
-----------

A list of current maintainers is
`on our website `__.

Code of conduct
---------------
Fairlearn follows the `Fairlearn Organization's Code of Conduct `__.

Issues
------

Regular (non-security) issues
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Issues are meant for bugs, feature requests, and documentation
improvements. Please submit a report through
`GitHub issues `__.
A maintainer will respond promptly as appropriate.

Maintainers will try to link duplicate issues when possible.

Reporting security issues
~~~~~~~~~~~~~~~~~~~~~~~~~

To report security issues please send an email to
``fairlearn-internal@python.org``.

.. |MIT license| image:: https://img.shields.io/badge/License-MIT-blue.svg
   :target: https://github.com/fairlearn/fairlearn/blob/main/LICENSE
.. |PyPI| image:: https://img.shields.io/pypi/v/fairlearn?color=blue
   :target: https://pypi.org/project/fairlearn/
.. |Discord| image:: https://img.shields.io/discord/840099830160031744
   :target: https://discord.gg/R22yCfgsRn
.. |StackOverflow| image:: https://img.shields.io/badge/StackOverflow-questions-blueviolet
   :target: https://stackoverflow.com/questions/tagged/fairlearn

Owner

Name: Fairlearn
Login: fairlearn
Kind: organization

Website: https://fairlearn.org
Twitter: fairlearn
Repositories: 4
Profile: https://github.com/fairlearn

Citation (CITATION.cff)

cff-version: 1.2.0
message: "If you use this software, please cite it as below."
authors:
  - given-names: "Fairlearn contributors"
title: "Fairlearn"
url: "https://fairlearn.org/"
preferred-citation:
  type: article
  authors:
    - given-names: "Hilde"
      family-names: "Weerts"
    - given-names: "Miroslav"
      family-names: "Dudík"
    - given-names: "Richard"
      family-names: "Edgar"
    - given-names: "Adrin"
      family-names: "Jalali"
    - given-names: "Roman"
      family-names: "Lutz"
    - given-names: "Michael"
      family-names: "Madaio"
  title: "Fairlearn: Assessing and Improving Fairness of AI Systems"
  year: 2023
  journal: "Journal of Machine Learning Research"
  url: "http://jmlr.org/papers/v24/23-0389.html"
  volume: 24
  number: 257
  pages: "1--8"

Committers

Last synced: 8 months ago

All Time

Total Commits: 940
Total Committers: 104
Avg Commits per committer: 9.038
Development Distribution Score (DDS): 0.689

Past Year

Commits: 105
Committers: 27
Avg Commits per committer: 3.889
Development Distribution Score (DDS): 0.61

Top Committers

Name	Email	Commits
Richard Edgar	r**r@m**m	292
Roman Lutz	r**z@m**m	176
Brandon Horn	r**n@m**m	82
Tamara	a**a@g**m	51
MiroDudik	m**k@g**m	43
Adrin Jalali	a**i@g**m	37
Roman Lutz	r**3@g**m	32
Tahar Allouche	t**o@g**m	23
Hilde Weerts	2****s	17
Allie Saizan	a**n@g**m	16
iofall	5****l	8
bethz	6****z	7
Mehrnoosh Sameki	4****i	6
Samuel Kalu	k**9@g**m	5
Nils	s**r@g**m	5
Janhavi Mahajan	j****3	4
vincent d warmerdam	v**m@g**m	4
rensoostenbach	3****h	4
SeanMcCarren	m**n@g**m	4
Niha Pereira	n**a@g**m	4
Kevin Robinson	k**0@g**m	4
Brenda Thng	5****g	3
Francesco Bruzzesi	4****i	3
Manojit Nandi	m**2@g**m	3
gaugup	4****p	3
dependabot[bot]	4****]	3
Stefanie Senger	9****r	3
Parul Gupta	p**5@g**m	3
Edoardo Abati	2****i	3
Emma Carballal	8****l	3
and 74 more...

Committer Domains (Top 20 + Academic)

microsoft.com: 6 live.it: 1 sussex.ac.uk: 1 gradientinstitute.org: 1 colorado.edu: 1 feldroy.com: 1 camilasan.com: 1 utep.edu: 1 elementai.com: 1 mdudik-sb.northamerica.corp.microsoft.com: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time

Total issues: 219
Total pull requests: 342
Average time to close issues: over 1 year
Average time to close pull requests: 2 months
Total issue authors: 64
Total pull request authors: 62
Average comments per issue: 4.9
Average comments per pull request: 2.55
Merged pull requests: 263
Bot issues: 0
Bot pull requests: 3

Past Year

Issues: 46
Pull requests: 161
Average time to close issues: about 1 month
Average time to close pull requests: 14 days
Issue authors: 11
Pull request authors: 27
Average comments per issue: 1.91
Average comments per pull request: 2.16
Merged pull requests: 125
Bot issues: 0
Bot pull requests: 2

View more stats

Top Authors

Issue Authors

romanlutz (34)
hildeweerts (32)
TamaraAtanasoska (27)
adrinjalali (17)
riedgar-ms (10)
MiroDudik (8)
alliesaizan (6)
taharallouche (6)
mmadaio (4)
pradeepdev-1995 (4)
LeJit (3)
kevinrobinson (3)
excepto64 (3)
haytham918 (3)
SeanMcCarren (3)

Pull Request Authors

TamaraAtanasoska (107)
taharallouche (45)
riedgar-ms (41)
adrinjalali (23)
hildeweerts (22)
romanlutz (22)
alliesaizan (15)
StefanieSenger (10)
Mariam-ke (10)
FBruzzesi (8)
EdAbati (8)
nnpereira (7)
eskayML (7)
naruminato1 (6)
SeanMcCarren (5)

Top Labels

Issue Labels

documentation (75) help wanted (59) enhancement (25) good first issue (22) maintenance (18) CI (14) question (10) dependencies (9) bug (8) easy (7) test (6) reserved-for-sprint (5) Waiting for OP's Response (3) api-reference (2) API (2) website (1) proposal (1) triage (1)

Pull Request Labels

documentation (17) No Changelog Needed (16) enhancement (11) dependencies (9) CI (5) spam (5) test (4) API (4) python (3) github_actions (2) bug (1) stale (1)

Packages

Total packages: 4
Total downloads:
- npm 5 last-month
- pypi 141,604 last-month
Total docker downloads: 340

Total dependent packages: 34
(may contain duplicates)
Total dependent repositories: 147
(may contain duplicates)
Total versions: 52
Total maintainers: 6

pypi.org: fairlearn

A Python package to assess and improve fairness of machine learning models.

Homepage: https://github.com/fairlearn/fairlearn
Documentation: https://fairlearn.readthedocs.io/
License: MIT License
Latest release: 0.12.0
published about 1 year ago

Versions: 22
Dependent Packages: 33
Dependent Repositories: 145
Downloads: 141,604 Last month
Docker Downloads: 340

Rankings

Dependent packages count: 0.5%

Downloads: 0.6%

Dependent repos count: 1.3%

Average: 1.5%

Stargazers count: 1.7%

Docker downloads count: 2.5%

Forks count: 2.8%

Maintainers (5)

adrin MiroDudik riedgar-ms romanlutz fairlearn-deploy

Last synced: 6 months ago

proxy.golang.org: github.com/fairlearn/fairlearn

Documentation: https://pkg.go.dev/github.com/fairlearn/fairlearn#section-documentation
License: mit
Latest release: v0.12.0
published about 1 year ago

Versions: 19
Dependent Packages: 0
Dependent Repositories: 0

Rankings

Dependent packages count: 5.4%

Average: 5.6%

Dependent repos count: 5.8%

Last synced: 6 months ago

npmjs.org: fairlearn-widget

Fairlearn widget for Jupyter notebooks

Homepage: https://github.com/fairlearn/fairlearn#readme
License: See license file 'LICENSE'
Latest release: 0.1.0
published over 6 years ago

Versions: 1
Dependent Packages: 1
Dependent Repositories: 1
Downloads: 5 Last month

Rankings

Forks count: 1.9%

Stargazers count: 2.1%

Dependent repos count: 10.8%

Average: 12.3%

Dependent packages count: 21.3%

Downloads: 25.4%

Maintainers (1)

brandonhorn

Last synced: 7 months ago

conda-forge.org: fairlearn

The fairlearn project seeks to enable anyone involved in the development of artificial intelligence (AI) systems to assess their system's fairness and mitigate the observed unfairness.

Homepage: https://github.com/fairlearn/fairlearn
License: MIT
Latest release: 0.7.0
published over 4 years ago

Versions: 10
Dependent Packages: 0
Dependent Repositories: 1

Rankings

Forks count: 9.0%

Stargazers count: 10.3%

Average: 23.7%

Dependent repos count: 24.1%

Dependent packages count: 51.5%

Last synced: 6 months ago

Dependencies

.github/workflows/build-test.yml actions

.github/workflows/linting.yml actions

actions/checkout v3 composite
actions/setup-python v3 composite
psf/black 23.9.1 composite

.github/workflows/nightly.yml actions

.github/workflows/release-wheel.yml actions

JesseTG/rm v1.0.3 composite
actions/checkout v3 composite
actions/download-artifact v3 composite
actions/setup-python v4 composite
actions/upload-artifact v3 composite
pypa/gh-action-pypi-publish release/v1 composite

.github/workflows/test-all-deps.yml actions

actions/checkout v3 composite
actions/setup-python v3 composite
codecov/codecov-action v3 composite

.github/workflows/test-minimal-deps.yml actions

actions/checkout v3 composite
actions/setup-python v3 composite
codecov/codecov-action v3 composite

.github/workflows/test-other-ml.yml actions

actions/checkout v3 composite
actions/setup-python v3 composite
codecov/codecov-action v3 composite

docs/_static/versions.json meteor

pyproject.toml pypi

requirements-dev.txt pypi

autopep8 * development
black ==23.9.1 development
flake8 * development
flake8-blind-except * development
flake8-builtins * development
flake8-copyright * development
flake8-docstrings * development
flake8-logging-format * development
flake8-rst-docstrings * development
numpydoc * development
packaging * development
pydata-sphinx-theme ==0.14.1 development
pypandoc * development
pytest ==7.4.2 development
pytest-cov * development
pytest-mock * development
pytest-mpl * development
requirements-parser * development
seaborn * development
sphinx ==7.2.6 development
sphinx ==7.1.2 development
sphinx-autodoc-typehints * development
sphinx-gallery * development
sphinxcontrib-bibtex * development
wheel * development
xlrd * development

requirements.txt pypi

numpy >=1.24.4
pandas >=2.0.3
scikit-learn >=1.2.1
scipy >=1.9.3

setup.py pypi

line.strip *