Data

Tools

Indexes

Applications

Experiments

Open Source Science

dbias

Detecting Bias and ensuring Fairness in AI solutions

https://github.com/dreji18/fairness-in-ai

Science Score: 67.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ✓
    CITATION.cff file
    Found CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ✓
    DOI references
    Found 1 DOI reference(s) in README
  • ✓
    Academic publication links
    Links to: arxiv.org, springer.com
  • ○
    Academic email domains
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (10.5%) to scientific vocabulary

Keywords

bias fariness nlp
Last synced: 6 months ago · JSON representation ·

Repository

Detecting Bias and ensuring Fairness in AI solutions

Basic Info
  • Host: GitHub
  • Owner: dreji18
  • License: mit
  • Language: Jupyter Notebook
  • Default Branch: main
  • Homepage:
  • Size: 8.54 MB
Statistics
  • Stars: 96
  • Watchers: 3
  • Forks: 16
  • Open Issues: 5
  • Releases: 0
Topics
bias fariness nlp
Created about 4 years ago · Last pushed about 3 years ago
Metadata Files
Readme License Citation

README.md

Dbias - Detecting Bias and ensuring Fairness in AI solutions

This package is used to detect and mitigate biases in NLP tasks. The model is an end-to-end framework that takes data into a raw form, preprocess it, detect the various types of biases and mitigate them. The output is the text that is free from bias.

Downloads CI CI CI

For more details, we would suggest reading the paper - https://link.springer.com/article/10.1007/s41060-022-00359-4 - International Journal of Data Science and Analytics (2022) - https://arxiv.org/abs/2207.03938 - KDD 2022 Workshop on Data Science and Artificial Intelligence for Responsible Recommendations (DS4RRS)

| Feature | Output | |---|---| | Text Debiasing | Returns debiased news recommendations with bias probability | | Bias Classification | Classifies whether a news article is biased or not with probability | | Bias Words/Phrases Recognition | Extract Biased words or phrases from the news fragment | | Bias masking | Returns the news fragment with biased words masked out |

Installation

Use the package manager pip to install Dbias.

bash pip install Dbias pip install https://huggingface.co/d4data/en_pipeline/resolve/main/en_pipeline-any-py3-none-any.whl

Usage

To de-bias a news article ```python from Dbias.text_debiasing import *

returns unbiased recommendations for a given sentence fragment.

run("Billie Eilish issues apology for mouthing an anti-Asian derogatory term in a resurfaced video.", show_plot = True) ``` drawing

To Classify a news article whether it's biased or not ```python from Dbias.bias_classification import *

returns classification label for a given sentence fragment.

classifier("Nevertheless, Trump and other Republicans have tarred the protests as havens for terrorists intent on destroying property.") ```

To Recognize the biased words/phrases ```python from Dbias.bias_recognition import *

returns extracted biased entities from a given sentence fragment

recognizer("Christians should make clear that the perpetuation of objectionable vaccines and the lack of alternatives is a kind of coercion.") ```

To Mask out the biased portions of a given sentence fragment ```python from Dbias.bias_masking import *

returns extracted biased entities from a given sentence fragment

masking("The fact that the abortion rate among American blacks is far higher than the rate for whites is routinely chronicled and mourned.") ```

Please find more examples in the notebook section.

About

This is a collective pipeline comprises of 3 Transformer models to de-bias/reduce amount of bias in news articles. The three models are: - An English sequence classification model, trained on the MBIC Dataset, to detect bias and fairness in sentences (news articles). This model was built on top of distilbert-base-uncased model and trained for 30 epochs with a batch size of 16, a learning rate of 5e-5, and a maximum sequence length of 512. - An Entity Recognition model, which is is trained on MBIC Dataset to recognize the biased word/phrases in a sentence. This model was built on top of roberta-base offered by Spacy transformers. - A Masked Language model, which is a Pretrained model on English language using a masked language modeling (MLM) objective.

Author

This model is part of the Research topic "Bias and Fairness in AI" conducted by Deepak John Reji, Shaina Raza, Chen Ding If you use this work (code, model or data),

Please cite our Research Paper

and please star at: Bias & Fairness in AI, (2022), GitHub repository, https://github.com/dreji18/Fairness-in-AI

License

MIT License

Owner

  • Name: Deepak John Reji
  • Login: dreji18
  • Kind: user
  • Location: Bangalore
  • Company: ERM

I am an NLP practitioner with experience in developing and structuring solutions for the data science environment

Citation (CITATION.cff) 

@article{raza2022dbias,
  title={Dbias: detecting biases and ensuring fairness in news articles},
  author={Raza, Shaina and Reji, Deepak John and Ding, Chen},
  journal={International Journal of Data Science and Analytics},
  pages={1--21},
  year={2022},
  publisher={Springer}
}
https://link.springer.com/article/10.1007/s41060-022-00359-4

GitHub Events

Total
  • Watch event: 16
  • Issue comment event: 3
  • Fork event: 4
Last Year
  • Watch event: 16
  • Issue comment event: 3
  • Fork event: 4

Issues and Pull Requests

Last synced: 9 months ago

All Time
  • Total issues: 4
  • Total pull requests: 3
  • Average time to close issues: N/A
  • Average time to close pull requests: 10 minutes
  • Total issue authors: 3
  • Total pull request authors: 3
  • Average comments per issue: 1.0
  • Average comments per pull request: 1.0
  • Merged pull requests: 1
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • FJLopezGarcia (2)
  • rajneesh407 (1)
  • Muhtasham (1)
Pull Request Authors
  • shainaraza (1)
  • sugatoray (1)
  • dreji18 (1)
Top Labels
Issue Labels
Pull Request Labels

Packages

  • Total packages: 1
  • Total downloads:
    • pypi 225 last-month
  • Total dependent packages: 1
  • Total dependent repositories: 1
  • Total versions: 14
  • Total maintainers: 2
pypi.org: dbias

Detect, Recognize and de-bias textual data.

  • Versions: 14
  • Dependent Packages: 1
  • Dependent Repositories: 1
  • Downloads: 225 Last month
Rankings
Stargazers count: 8.6%
Dependent packages count: 10.1%
Downloads: 10.3%
Forks count: 10.9%
Average: 12.3%
Dependent repos count: 21.6%
Maintainers (2)
dreji18 shainaraza
Last synced: 6 months ago

Dependencies

Dbias.egg-info/requires.txt pypi
  • numpy ==1.19.5
  • pandas ==1.2.4
  • spacy ==3.2.1
  • tensorflow ==2.4.1
  • transformers ==4.6.1
requirements.txt pypi
  • numpy ==1.19.5
  • pandas ==1.2.4
  • spacy ==3.2.1
  • tensorflow ==2.4.1
  • transformers ==4.6.1
setup.py pypi