235-causal-guided-active-learning-for-debiasing-large-language-models

https://github.com/szu-advtech-2024/235-causal-guided-active-learning-for-debiasing-large-language-models

Scientific Fields

Repository

https://github.com/SZU-AdvTech-2024/235-Causal-Guided-Active-Learning-for-Debiasing-Large-Language-Models/blob/main/

# Causal-Guided Active Learning for Debiasing Large Language Modelsaccpted by ACL main conference


  


Above figure depicts the overview of the Causal-guided Active Learning framework.

## Dependencies

- Dependencies can be installed using `requirements.txt` and python3.8.

## Quick start for CAL

- Download llama2-13b-chat, vicuna-13b-v1.5 model and put them under ../../model/llama2-13b-chat and ../../model/vicuna-13b-v1.5

- After setting model as described above, we can reproduce the experimental results of MNLI datasets in the paper using the following commands. (For other LLMs and datasets, the commands are similar.)

### Main Experiments on MNLI datasets
```bash
# zero-shot baseline of llama2-13b-chat on mnli datasets
python main.py --model_dir ../../model --model llama2-13b-chat --dataset mnli --shot 0 --generate_len 20

# zero-shot CAL of llama2-13b-chat on mnli datasets
python main.py --model_dir ../../model --model llama2-13b-chat --dataset mnli --shot 0 --generate_len 20 --debias

# few-shot baseline of llama2-13b-chat on mnli datasets
python main.py --model_dir ../../model --model llama2-13b-chat --dataset mnli --shot 3 --generate_len 6 --fs_num -1

# few-shot CAL of llama2-13b-chat on mnli datasets
python main.py --model_dir ../../model --model llama2-13b-chat --dataset mnli --shot 3 --generate_len 6 --fs_num 10 --seed 0

# zero-shot baseline of GPT-4 on mnli datasets
python main.py --model_dir ../../model --model gpt4 --dataset mnli --shot 0 --generate_len 20 

# zero-shot CAL of GPT-4 on mnli datasets
python main.py --model_dir ../../model --model gpt4 --dataset mnli --shot 0 --generate_len 20 --debias
```

If you want to reproduce our biased data identification and bias pattern induction process or find new bias patterns of LLMs, you can install the python package [autobiasdetector](https://pypi.org/project/autobiasdetector/) that we specifically designed for this goal. (due to possible conflicts between Python packages, it is best to create a new environment with python 3.10) The example code for using the package [autobiasdetector](https://pypi.org/project/autobiasdetector/) is located in the biasdetect folder.

Please create an issue or contact me(https://zhouhaosun.github.io/) for any clarifications or suggestions.

Owner

Citation (citation.txt)

@inproceedings{REPO235,
    author = "Sun, Zhouhao and Du, Li and Ding, Xiao and Ma, Yixuan and Zhao, Yang and Qiu, Kaitao and Liu, Ting and Qin, Bing",
    editor = "Ku, Lun-Wei and Martins, Andre and Srikumar, Vivek",
    booktitle = "Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    month = "August",
    pages = "14455--14469",
    title = "{Causal-Guided Active Learning for Debiasing Large Language Models}",
    year = "2024"
}

GitHub Events