Data

Tools

Indexes

Applications

Experiments

Open Source Science

fairmetrics: An R package for group fairness evaluation

fairmetrics: An R package for group fairness evaluation - Published in JOSS (2025)

https://github.com/jianhuig/fairmetrics

Science Score: 95.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ✓
    DOI references
    Found 4 DOI reference(s) in README and JOSS metadata
  • ✓
    Academic publication links
    Links to: arxiv.org, joss.theoj.org, zenodo.org
  • ✓
    Committers with academic emails
    3 of 8 committers (37.5%) from academic institutions
  • ○
    Institutional organization owner
  • ✓
    JOSS paper metadata
    Published in Journal of Open Source Software
Last synced: 5 months ago · JSON representation

Repository

Basic Info
Statistics
  • Stars: 3
  • Watchers: 1
  • Forks: 1
  • Open Issues: 2
  • Releases: 7
Created about 2 years ago · Last pushed 5 months ago
Metadata Files
Readme Contributing License

README.md

fairmetrics: Fairness evaluation metrics with confidence intervals for binary protected attributes

CRAN_Status_Badge R-CMD-check License Downloads total arXiv JOSS DOI

A collection of functions for computing fairness metrics for machine learning and statistical models, including confidence intervals for each metric. The package supports the evaluation of group-level fairness criterion commonly used in fairness research, particularly in healthcare for binary protected attributes. It is based on the overview of fairness in machine learning written by Gao et al (2024) (https://arxiv.org/abs/2406.09307).

  • Link to online tutorial.
  • Link to preprint.

Installation

To install the latest CRAN release run:

r install.packages("fairmetrics")

To install the package from the Github repository run:

r devtools::install_github("jianhuig/fairmetrics")

Citation

To cite package ‘fairmetrics’ in publications use:

Gao J, Smith B, Chou B, Gronsbell J (2025). fairmetrics: Fairness Evaluation Metrics with Confidence Intervals. https://github.com/jianhuig/fairmetrics.

Smith, Gao, and Gronsbell (2025). fairmetrics: An R package for group fairness evaluation. arXiv:2506.06243.

Gao et al. (2024). What is Fair? Defining Fairness in Machine Learning for Health. arXiv:2406.09307.

A BibTeX entry for LaTeX users is

``` bibentry( bibtype = "Manual", title = "fairmetrics: Fairness Evaluation Metrics with Confidence Intervals for Binary Protected Attributes", author = c( person("Jianhui", "Gao"), person("Benjamin", "Smith"), person("Benson", "Chou"), person("Jessica", "Gronsbell") ), year = "2025", url = "https://github.com/jianhuig/fairmetrics" )

bibentry( bibtype = "Misc", key = "SmithGaoGronsbell_2025", title = "fairmetrics: An R package for group fairness evaluation", author = c( person("Benjamin", "Smith"), person("Jianhui", "Gao"), person("Jessica", "Gronsbell") ), year = "2025", month = "jun", note = "arXiv:2506.06243", url = "https://arxiv.org/abs/2506.06243", textVersion = "Smith, Gao, and Gronsbell (2025). fairmetrics: An R package for group fairness evaluation. arXiv:2506.06243."

)

bibentry( bibtype = "Misc", key = "GaoChouMcCawThurstonVargheseHongGronsbell_2024", title = "What is Fair? Defining Fairness in Machine Learning for Health", author = c( person("Jianhui", "Gao"), person("Benson", "Chou"), person("Zachary R.", "McCaw"), person("Hilary", "Thurston"), person("Paul", "Varghese"), person("Chuan", "Hong"), person("Jessica", "Gronsbell") ), year = "2024", month = "jun", note = "arXiv:2406.09307", url = "https://arxiv.org/abs/2406.09307", textVersion = "Gao et al. (2024). What is Fair? Defining Fairness in Machine Learning for Health. arXiv:2406.09307." ) ```

Similar Works

References

  1. Gao, J. et al. What is Fair? Defining Fairness in Machine Learning for Health. arXiv.org https://arxiv.org/abs/2406.09307 (2024).

Owner

  • Name: Jianhui Gao
  • Login: jianhuig
  • Kind: user
  • Location: Toronto
  • Company: University of Toronto

PhD student in statistics at the University of Toronto. Interested in semi-supervised learning and statistical genetics.

JOSS Publication

fairmetrics: An R package for group fairness evaluation
Published
September 24, 2025
DOI
Volume 10, Issue 113, Page 8497
Authors
Benjamin Smith ORCID
Department of Statistical Sciences, University of Toronto
Jianhui Gao ORCID
Department of Statistical Sciences, University of Toronto
Benson Chou ORCID
Department of Statistical Sciences, University of Toronto
Jessica Gronsbell ORCID
Department of Statistical Sciences, University of Toronto
Editor
Nikoleta Glynatsi ORCID
Tags
Fairness Machine Learning Software
View PDF Review Thread Software Archive

GitHub Events

Total
  • Create event: 5
  • Release event: 3
  • Issues event: 18
  • Watch event: 3
  • Issue comment event: 36
  • Public event: 1
  • Push event: 117
  • Pull request event: 1
  • Fork event: 1
Last Year
  • Create event: 5
  • Release event: 3
  • Issues event: 18
  • Watch event: 3
  • Issue comment event: 36
  • Public event: 1
  • Push event: 117
  • Pull request event: 1
  • Fork event: 1

Committers

Last synced: 5 months ago

All Time
  • Total Commits: 370
  • Total Committers: 8
  • Avg Commits per committer: 46.25
  • Development Distribution Score (DDS): 0.614
Past Year
  • Commits: 239
  • Committers: 6
  • Avg Commits per committer: 39.833
  • Development Distribution Score (DDS): 0.402
Top Committers
Name Email Commits
benyamindsmith b****h@g****m 143
Jianhui Gao 5****g@u****m 79
Benson-chou b****u@m****a 39
benyamindsmith b****n@a****m 38
jianhuig j****o@m****a 34
Jesse Gronsbell 7****s@u****m 19
Benjamin Smith 4****h@u****m 17
Nikoleta-v3 g****i@e****e 1
Committer Domains (Top 20 + Academic)
mail.utoronto.ca: 2 evolbio.mpg.de: 1 alterclimate.com: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 14
  • Total pull requests: 1
  • Average time to close issues: 28 days
  • Average time to close pull requests: N/A
  • Total issue authors: 3
  • Total pull request authors: 1
  • Average comments per issue: 2.79
  • Average comments per pull request: 0.0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 14
  • Pull requests: 1
  • Average time to close issues: 28 days
  • Average time to close pull requests: N/A
  • Issue authors: 3
  • Pull request authors: 1
  • Average comments per issue: 2.79
  • Average comments per pull request: 0.0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • vankesteren (8)
  • benyamindsmith (5)
  • StevenGolovkine (1)
Pull Request Authors
  • Nikoleta-v3 (1)
Top Labels
Issue Labels
enhancement (3) help wanted (2) documentation (1)
Pull Request Labels

Packages

  • Total packages: 1
  • Total downloads:
    • cran 369 last-month
  • Total dependent packages: 0
  • Total dependent repositories: 0
  • Total versions: 7
  • Total maintainers: 1
cran.r-project.org: fairmetrics

Fairness Evaluation Metrics with Confidence Intervals for Binary Protected Attributes

  • Versions: 7
  • Dependent Packages: 0
  • Dependent Repositories: 0
  • Downloads: 369 Last month
Rankings
Dependent packages count: 26.5%
Forks count: 29.0%
Stargazers count: 29.5%
Dependent repos count: 32.6%
Average: 40.9%
Downloads: 86.7%
Maintainers (1)
benyamin.smith@mail.utoronto.ca
Last synced: 5 months ago

Dependencies

.github/workflows/pkgdown.yaml actions
  • JamesIves/github-pages-deploy-action v4.5.0 composite
  • actions/checkout v4 composite
  • r-lib/actions/setup-pandoc v2 composite
  • r-lib/actions/setup-r v2 composite
  • r-lib/actions/setup-r-dependencies v2 composite
DESCRIPTION cran
  • R >= 2.10 depends
  • dplyr * imports
  • magrittr * imports
  • knitr * suggests
  • rmarkdown * suggests
  • testthat >= 3.0.0 suggests