Data

Tools

Indexes

Applications

Experiments

Open Source Science

fairvis

FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning

https://github.com/poloclub/fairvis

Science Score: 49.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ✓
    DOI references
    Found 1 DOI reference(s) in README
  • ✓
    Academic publication links
    Links to: arxiv.org
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (14.3%) to scientific vocabulary
Last synced: 7 months ago · JSON representation

Repository

FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning

Basic Info
Statistics
  • Stars: 38
  • Watchers: 7
  • Forks: 7
  • Open Issues: 1
  • Releases: 0
Created almost 7 years ago · Last pushed almost 2 years ago
Metadata Files
Readme License Citation

README.md

FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning

FairVis is a visual analytics system that allows users to audit their classification models for intersectional bias. Users can generate subgroups of their data and investigate if a model is underperforming for certain populations.

FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning
ngel Alexander Cabrera, Will Epperson, Fred Hohman, Minsuk Kahng, Jamie Morgenstern, Duen Horng (Polo) Chau IEEE Conference on Visual Analytics Science and Technology (VAST). 2019.

teaser figure

In this example we show how FairVis can be used on the COMPAS dataset to find significant disparities in false positive rates between African American and Caucasian defendants that are not supported by base rates. The primary components of the system are the following:

A. View distributions of the dataset's features and generate subgroups.

B. Visualize subgroup performance in relation to selected metrics.

C. Compare selected subgroups and view details.

D. Find suggested underperforming subgroups and similar groups.

For more details about the system and its use cases, see the IEEE VAST paper.

Installation

Clone the repository:

git clone https://github.com/poloclub/FairVis.git

Then initialize the React project by running

npm install

Usage

Run the server with

npm start

Adding a new dataset

  1. Run a model on your data and and create a new file with the last two columns being the output class (between 0-1) of the model and the ground truth labels (0 or 1). Note that only binary classification is currently supported. Examples of models in Jupyter Notebook format can be found in ./models.

  2. Run the preprocess.py script on your classified data, e.g. python3 preprocess.py my-data-with-classes.csv. Additional options for the helper function can be found using python3 preprocess.py -h.

  3. Save the processed file to ./src/data/.

  4. Import the file in the src/components/Welcome.js component.

  5. Add a new row to the table in Welcome.js around line 140 in the form of the other datsets.

Researchers

| Name | Affiliation | |-----------------------|---------------------------------| | ngel Alexander Cabrera | Georgia Tech | | Will Epperson | Georgia Tech | | Fred Hohman | Georgia Tech | | Minsuk Kahng | Georgia Tech | | Jamie Morgenstern | Georgia Tech | | Duen Horng (Polo) Chau | Georgia Tech |

Citation

@inproceedings{cabrera2019fairvis, title={FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning}, author={Cabrera, {'A}ngel Alexander and Epperson, Will and Hohman, Fred and Kahng, Minsuk and Morgenstern, Jamie and Chau, Duen Horng}, booktitle={2019 IEEE Conference on Visual Analytics Science and Technology (VAST)}, pages={46-56}, year={2019}, publisher={IEEE}, doi={10.1109/VAST47406.2019.8986948}, url={https://cabreraalex.com/#/paper/fairvis} }

License

MIT License. See LICENSE.md.

Owner

  • Name: Polo Club of Data Science
  • Login: poloclub
  • Kind: organization
  • Location: Atlanta, GA

Research project repositories, from the Polo Club of Data Science at Georgia Tech

GitHub Events

Total
  • Watch event: 3
  • Fork event: 1
Last Year
  • Watch event: 3
  • Fork event: 1

Committers

Last synced: 11 months ago

All Time
  • Total Commits: 14
  • Total Committers: 5
  • Avg Commits per committer: 2.8
  • Development Distribution Score (DDS): 0.429
Past Year
  • Commits: 0
  • Committers: 0
  • Avg Commits per committer: 0.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
Alex Cabrera a****a@g****m 8
kevinrobinson k****0@g****m 2
Fred Hohman f****n@g****m 2
Will Epperson w****p@l****m 1
Minsuk Kahng m****g 1

Issues and Pull Requests

Last synced: 11 months ago

All Time
  • Total issues: 0
  • Total pull requests: 3
  • Average time to close issues: N/A
  • Average time to close pull requests: about 2 months
  • Total issue authors: 0
  • Total pull request authors: 2
  • Average comments per issue: 0
  • Average comments per pull request: 1.0
  • Merged pull requests: 1
  • Bot issues: 0
  • Bot pull requests: 1
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
Pull Request Authors
  • dependabot[bot] (2)
  • kevinrobinson (2)
Top Labels
Issue Labels
Pull Request Labels
dependencies (2)

Dependencies

package-lock.json npm
  • 1479 dependencies
package.json npm
  • gh-pages ^2.0.1 development
  • @material-ui/core ^3.9.2
  • @material-ui/icons ^3.0.2
  • @material-ui/lab ^3.0.0-alpha.30
  • d3 ^5.9.1
  • d3-tip ^0.9.1
  • react ^16.8.1
  • react-dom ^16.8.5
  • react-ga ^2.6.0
  • react-icons ^3.7.0
  • react-scripts ^3.1.1
  • react-select ^2.4.2
  • react-vega-lite ^2.0.2
  • typescript ^3.4.4
  • vega-lib ^4.4.0
  • vega-lite ^2.6.0