Measuring Biases in Masked Language Models for PyTorch Transformers

Evaluate biases in pre-trained or re-trained masked language models (MLMs), such as those available through HuggingFace. This package computes bias scores across various bias types, using benchmark datasets like CrowS-Pairs (CPS) and StereoSet (SS) (intrasentence), or custom datasets. You can also compare relative bias between two MLMs, or evaluate re-trained MLMs versus their pre-trained base models.

Evaluation Methods

Bias scores for an MLM are computed for sentence pairs in the dataset using measures that represent MLM preference (or prediction quality). Bias against disadvantaged groups for a sentence pair is represented by a higher relative measure value for a sentence in adv compared to dis.

Iterative Masking Experiment (IME): For each sentence, an MLM masks one token at a time until all tokens are masked once, generating n logits or predictions for a sentence with n tokens.

Measures

We use state-of-the-art measures computed under the IME:

• CRR: Difference in reciprocal rank of a predicted token (always equal to 1) and the reciprocal rank of a masked token arXiv
• CRRA: CRR with Attention weights arXiv
• ΔP: Difference in log-liklihood of a predicted token and the masked token arXiv
• ΔPA: ΔP with Attention weights arXiv

Pseudo log-likelihood measures computed with a single encoded input (see References for more details): - CSPS: CrowS-Pairs Score is a score for an MLM selecting unmodified tokens given modified ones arXiv - SSS: StereoSet Score is a score for an MLM selecting modified tokens given unmodified ones arXiv - AUL: All Unmasked Likelihood is a score generated by predicting all tokens in a single unmasked input arXiv - AULA: AUL with Attention weights arXiv

Setup

bash pip install mlm-bias

```python import mlm_bias

Load and sample the CPS benchmark dataset

cpsdataset = mlmbias.BiasBenchmarkDataset("cps") cps_dataset.sample(indices=list(range(10)))

Specify the model

model = "bert-base-uncased"

Initialize the BiasMLM evaluator

mlmbiasevaluator = mlmbias.BiasMLM(model, cpsdataset)

Evaluate the model (can pass batching=True for larger samples)

result = mlmbiasevaluator.evaluate(attention=True)

Print the results, including total bias scores and scores by category for each measure

def formatmlmbiasresults(result, totalonly=False, sep="\n"): outl = [] if totalonly: for measure in result['biasscores'].keys(): outl.append((f"{measure.replace('d','Δ').upper()} " \ f"total = {round(result['biasscores'][measure]['total'],3)}")) else: for measure in result['biasscores'].keys(): outl.append(f"Measure = {measure.replace('d','Δ').upper()}") for biastype, score in result['bias_scores'][measure].items(): outl.append(f"- {biastype} = {round(score,3)}") return f"{sep}".join(outl) print(formatmlmbias_results(result))

Save the results

result.save("./mlm-biascpsbert-base-uncased") ```

Example Script

Clone the repository and install the package:

bash git clone https://github.com/zalkikar/mlm-bias.git cd mlm-bias python3 -m pip install .

Run the mlm_bias.py example script:

```bash usage: mlmbias.py [-h] --data {cps,ss,custom} --modelnameorpath MODELNAMEOR_PATH [--modelnameorpath2 MODELNAMEORPATH2] [--output OUTPUT] [--measures {all,crr,crra,dp,dpa,aul,aula,csps,sss}] [--start START] [--end END] [--batching]

options: -h, --help show this help message and exit --data {cps,ss,custom} Paired sentences from benchmark or supplied line by line dataset in /data directory. Provide bias types in "/biastypes.txt" and biased sentences in "/dis.txt" and "/adv.txt" accordingly. --modelnameorpath MODELNAMEORPATH Model (MLM) to compute bias measures for. Must be supported by HuggingFace. --modelnameorpath2 MODELNAMEORPATH_2 Model (MLM) to compute bias measures for. Must be supported by HuggingFace. Used to compare with "--model" --output OUTPUT Full path (eg. dir/file.txt) for output directory with computed measures. --measures {all,crr,crra,dp,dpa,aul,aula,csps,sss} Measures computed to evaluate bias in MLMs. --start START Start index of dataset sample. --end END End index of dataset sample. --batching Batched inputs. ```

Example arguments:

```bash

Single MLM

python3 mlmbias.py --data cps --modelnameorpath roberta-base --start 0 --end 30 --batching python3 mlmbias.py --data ss --modelnameorpath bert-base-uncased --start 0 --end 30 --batching

Relative between two MLMs

python3 mlmbias.py --data cps --modelnameorpath roberta-base --start 0 --end 30 --modelnameorpath2 bert-base-uncased --batching ```

Output directories (default arguments): - /data contains cps.csv (CPS) and/or ss.csv (SS). - /eval contains out.txt with computed bias scores and pickled result objects.

Example Output:

bash python3 mlm_bias.py --data cps --model_name_or_path bert-base-uncased --start 0 --end 30 --batching

```bash Created output directory. Created Data Directory |██████████████████████████████| 1/1 [100%] in 0s ETA: 0s Downloaded Data [CrowSPairs] |██████████████████████████████| 1/1 [100%] in 0s ETA: 0s Loaded Data [CrowSPairs] |██████████████████████████████| 1/1 [100%] in 0s ETA: 0s Evaluating Bias [bert-base-uncased] |██████████████████████████████| 30/30 [100%] in 31s ETA: 0s Saved bias results for bert-base-uncased in ./eval/bert-base-uncased

Saved scores in ./eval/out.txt

MLM: bert-base-uncased CRR total = 76.667 CRRA total = 60.0 ΔP total = 63.333 ΔPA total = 63.333 AUL total = 63.333 AULA total = 66.667 SSS total = 50.0 CSPS total = 63.333 ```

In this example, ./eval/out.txt also contains bias scores by category for each measure.

Custom Datasets

Compute bias scores for a custom dataset directory with the following line-by-line files:

• bias_types.txt containing bias categories.
• dis.txt and adv.txt containing sentence pairs, where:
  • dis.txt contains sentences with bias against disadvantaged groups (stereotypical) and
  • adv.txt contains sentences with bias against advantaged groups (anti-stereotypical).

Citation

If using this for research, please cite the following:

bibtex @inproceedings{zalkikar-chandra-2025-measuring, title = "Measuring Social Biases in Masked Language Models by Proxy of Prediction Quality", author = "Zalkikar, Rahul and Chandra, Kanchan", editor = "Che, Wanxiang and Nabende, Joyce and Shutova, Ekaterina and Pilehvar, Mohammad Taher", booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)", month = jul, year = "2025", address = "Vienna, Austria", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2025.acl-long.68/", pages = "1337--1361", ISBN = "979-8-89176-251-0" }

References

bibtex @article{Kaneko_Bollegala_2022, title={Unmasking the Mask – Evaluating Social Biases in Masked Language Models}, volume={36}, url={https://ojs.aaai.org/index.php/AAAI/article/view/21453}, DOI={10.1609/aaai.v36i11.21453}, number={11}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Kaneko, Masahiro and Bollegala, Danushka}, year={2022}, month={Jun.}, pages={11954-11962} }

bibtex @InProceedings{10.1007/978-3-031-33374-3_42, author="Salutari, Flavia and Ramos, Jerome and Rahmani, Hossein A. and Linguaglossa, Leonardo and Lipani, Aldo", editor="Kashima, Hisashi and Ide, Tsuyoshi and Peng, Wen-Chih", title="Quantifying the Bias of Transformer-Based Language Models for African American English in Masked Language Modeling", booktitle="Advances in Knowledge Discovery and Data Mining", year="2023", publisher="Springer Nature Switzerland", address="Cham", pages="532--543", isbn="978-3-031-33374-3" }

bibtex @inproceedings{nangia-etal-2020-crows, title = "{C}row{S}-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models", author = "Nangia, Nikita and Vania, Clara and Bhalerao, Rasika and Bowman, Samuel R.", editor = "Webber, Bonnie and Cohn, Trevor and He, Yulan and Liu, Yang", booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)", month = nov, year = "2020", address = "Online", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2020.emnlp-main.154", doi = "10.18653/v1/2020.emnlp-main.154", pages = "1953--1967" }

bibtex @inproceedings{nadeem-etal-2021-stereoset, title = "{S}tereo{S}et: Measuring stereotypical bias in pretrained language models", author = "Nadeem, Moin and Bethke, Anna and Reddy, Siva", editor = "Zong, Chengqing and Xia, Fei and Li, Wenjie and Navigli, Roberto", booktitle = "Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)", month = aug, year = "2021", address = "Online", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2021.acl-long.416", doi = "10.18653/v1/2021.acl-long.416", pages = "5356--5371" }