Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/kundajelab/abstention

Algorithms for abstention, calibration and domain adaptation to label shift.

https://github.com/kundajelab/abstention

Science Score: 10.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ○
    codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ✓
    Academic publication links
    Links to: arxiv.org
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (6.5%) to scientific vocabulary

Keywords

abstention calibration domain-adaptation label-shift prior-probability-shift
Last synced: 6 months ago · JSON representation

Repository

Algorithms for abstention, calibration and domain adaptation to label shift.

Basic Info
  • Host: GitHub
  • Owner: kundajelab
  • Language: Python
  • Default Branch: master
  • Homepage:
  • Size: 7.47 MB
Statistics
  • Stars: 37
  • Watchers: 8
  • Forks: 4
  • Open Issues: 0
  • Releases: 0
Topics
abstention calibration domain-adaptation label-shift prior-probability-shift
Created over 8 years ago · Last pushed over 5 years ago
Metadata Files
Readme

README.md

Abstention, Calibration & Label Shift

Algorithms for abstention, calibration and domain adaptation to label shift.

Associated papers:

Shrikumar A*†, Alexandari A*, Kundaje A†, A Flexible and Adaptive Framework for Abstention Under Class Imbalance

Alexandari A*, Kundaje A†, Shrikumar A*†, Maximum Likelihood with Bias-Corrected Calibration is Hard-To-Beat at Label Shift Adaptation

*co-first authors † co-corresponding authors

Examples

See https://github.com/blindauth/abstention_experiments and https://github.com/blindauth/labelshiftexperiments for colab notebooks reproducing the experiments in the papers.

Installation

pip install abstention

Algorithms implemented

For calibration: - Platt Scaling - Isotonic Regression - Temperature Scaling - Vector Scaling - Bias-Corrected Temperature Scaling - No-Bias Vector Scaling

For domain adaptation to label shift: - Expectation Maximization (Saerens et al., 2002) - Black-Box Shift Learning (BBSL) (Lipton et al., 2018) - Regularized Learning under Label Shifts (RLLS) (Azizzadenesheli et al., 2019)

For abstention: - Metric-specific abstention methods described in A Flexible and Adaptive Framework for Abstention Under Class Imbalance, including abstention to optimize auROC, auPRC, sensitivity at a target specificity and weighted Cohen's Kappa - Jensen-Shannon Divergence from class priors - Entropy in the predicted class probabilities (Wan, 1990) - Probability of the highest-predicted class (Hendrycks & Gimpel, 2016) - The method of Fumera et al., 2000 - See Colab notebook experiments in https://github.com/blindauth/abstention_experiments for details on how to use the various abstention methods.

Contact

If you have any questions, please contact:

Avanti Shrikumar: avanti [dot] shrikumar [at] gmail.com

Amr Alexandari: amr [dot] alexandari [at] gmail.com

Anshul Kundaje: akundaje [at] stanford [dot] edu

Owner

  • Name: Kundaje Lab
  • Login: kundajelab
  • Kind: organization
  • Location: Stanford University

Compbio and machine learning code repositories from the Kundaje Lab at Stanford Genetics and Computer Science Depts.

GitHub Events

Total
  • Watch event: 1
Last Year
  • Watch event: 1

Committers

Last synced: 7 months ago

All Time
  • Total Commits: 110
  • Total Committers: 5
  • Avg Commits per committer: 22.0
  • Development Distribution Score (DDS): 0.182
Past Year
  • Commits: 0
  • Committers: 0
  • Avg Commits per committer: 0.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
Avanti Shrikumar a****r@g****m 90
amrmx a****d@g****m 8
alexandari a****i@g****m 6
Amr a****i@u****m 5
Avanti Shrikumar a****i@s****t 1
Committer Domains (Top 20 + Academic)
sh-101-58.int: 1

Issues and Pull Requests

Last synced: 8 months ago

All Time
  • Total issues: 0
  • Total pull requests: 2
  • Average time to close issues: N/A
  • Average time to close pull requests: about 23 hours
  • Total issue authors: 0
  • Total pull request authors: 1
  • Average comments per issue: 0
  • Average comments per pull request: 0.0
  • Merged pull requests: 2
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
Pull Request Authors
  • AvantiShri (2)
Top Labels
Issue Labels
Pull Request Labels

Packages

  • Total packages: 1
  • Total downloads:
    • pypi 2,373 last-month
  • Total dependent packages: 1
  • Total dependent repositories: 1
  • Total versions: 4
  • Total maintainers: 1
pypi.org: abstention

Functions for abstention, calibration and label shift domain adaptation

  • Versions: 4
  • Dependent Packages: 1
  • Dependent Repositories: 1
  • Downloads: 2,373 Last month
Rankings
Dependent packages count: 4.8%
Stargazers count: 11.0%
Average: 13.8%
Forks count: 15.4%
Downloads: 16.5%
Dependent repos count: 21.6%
Maintainers (1)
AvantiShri
Last synced: 7 months ago

Dependencies

abstention.egg-info/requires.txt pypi
  • numpy >=1.9
  • scikit-learn >=0.20.0
  • scipy >=1.1.0
setup.py pypi
  • numpy >=1.9
  • scikit-learn >=0.20.0
  • scipy >=1.1.0