https://github.com/amazon-science/aws-swing
Science Score: 26.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
○DOI references
-
○Academic publication links
-
○Committers with academic emails
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (9.5%) to scientific vocabulary
Keywords from Contributors
Repository
Basic Info
- Host: GitHub
- Owner: amazon-science
- License: apache-2.0
- Language: Python
- Default Branch: main
- Size: 40 KB
Statistics
- Stars: 10
- Watchers: 2
- Forks: 2
- Open Issues: 2
- Releases: 0
Metadata Files
README.md
SWING 🏌️: Balancing Coverage and Faithfulness for Dialogue Summarization
Authors: Kung-Hsiang Huang (khhuang3@illinois.edu), Siffi Singh, Xiaofei Ma, Wei Xiao, Feng Nan, Nicholas Dingwall, William Yang Wang, Kathleen McKeown .
Dependencies
First, create a virtual environment and install depednencies specified in requirements.txt
conda create -n ds python=3.8
conda activate ds
pip install -r requirements.txt
Then, create separate enviroments for BARTScore and FactCC, following the instructions for BARTScore and FactCC.
Data
The preprocessed data can be downloaded from here (dialogsum.zip and samsum.zip). Please create a data folder and unzip these two files into this folder.
Training
To train the model, run train.py. For example,
python train.py --exp_name $EXP_NAME --model_name facebook/bart-large --learning_rate 3e-5 --weight_decay 1e-3 --warmup_epoch 0 --accumulate_step 4 --batch_size 2 --dataset dialogsum --use_nli --do_uncovered --do_invalid --uncovered_weights 0.7 --invalid_weights 0.2 --do_factcc_validate --do_gradient_checkpointing
Training parameters are specified in args.py. You can specify each the value of each argskey by passing `--argskey arg_value`. Below illustrate some of the important keys.
``` --maxsequencelength: Maximum input length. Dialogues longer than this length will be truncated.
--model_name: The name of the model to load from HugingFace.
--dataset: One of {dialogsum, samsum}.
--use_nli: Enable this will train a generator with the NLIBART class, which is also proposed model.
--do_invalid: Do contrastive learning. (Invalid loss is the name we gave in the early stage of the experiment)
--do_uncovered: Do uncovered loss.
--expname: Name of the experiment. The model checkpoint will be saved in `args.outputdir/args.exp_name`.
--data_dir: (Deprecated) Directory of the input data. Specifying --dataset would affect this parameter.
--use_robust: (Deprecated) Do MLE with adversarial training. This was used in the early stage of the experiment.
--dofactccvalidate: (Deprecated) Use FactCC to further validate the goodness of the generated summary. Not used in the final solution.
--dofactccuncovered: (Deprecated) Not used in the final solution. ```
The trained checkpoints can be found in here ([dialogsum|samsum]_best/best.pt) for research purposes.
Evaluation
To run evaluation on trained models, execute the test.py script as follows:
python test.py --checkpoint_path $PATH_TO_MODEL/best.pt
If you already have your generated summaries (e.g. our training script would produce a $PATH_TO_OUTOUT/test_pred.json), you can directly run the following command to avoid running inference again and save time.
python test_file.py --dataset samsum --output_file $PATH_TO_OUTOUT/test_pred.json
Citation
bibtex
@inproceedings{huang-etal-2023-swing,
title = "SWING 🏌️: Balancing Coverage and Faithfulness for Dialogue Summarization",
author = "Huang, Kung-Hsiang and
Singh, Siffi and
Ma, Xiaofei and
Xiao, Wei and
Nan, Feng and
Dingwall, Nicholas and
Wang, William Yang and
McKeown, Kathleen",
booktitle = "Findings of the Association for Computational Linguistics: EACL 2023",
year = "2023",
publisher = "Association for Computational Linguistics",
}
Owner
- Name: Amazon Science
- Login: amazon-science
- Kind: organization
- Website: https://amazon.science
- Twitter: AmazonScience
- Repositories: 80
- Profile: https://github.com/amazon-science
GitHub Events
Total
- Delete event: 1
- Issue comment event: 1
- Pull request event: 3
- Create event: 3
Last Year
- Delete event: 1
- Issue comment event: 1
- Pull request event: 3
- Create event: 3
Committers
Last synced: about 1 year ago
Top Committers
| Name | Commits | |
|---|---|---|
| dependabot[bot] | 4****] | 5 |
| Siffi Singh | s****6@g****m | 4 |
| Amazon GitHub Automation | 5****o | 1 |
Issues and Pull Requests
Last synced: 11 months ago
All Time
- Total issues: 0
- Total pull requests: 9
- Average time to close issues: N/A
- Average time to close pull requests: 2 months
- Total issue authors: 0
- Total pull request authors: 1
- Average comments per issue: 0
- Average comments per pull request: 0.22
- Merged pull requests: 4
- Bot issues: 0
- Bot pull requests: 9
Past Year
- Issues: 0
- Pull requests: 5
- Average time to close issues: N/A
- Average time to close pull requests: 3 months
- Issue authors: 0
- Pull request authors: 1
- Average comments per issue: 0
- Average comments per pull request: 0.4
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 5
Top Authors
Issue Authors
Pull Request Authors
- dependabot[bot] (12)
Top Labels
Issue Labels
Pull Request Labels
Dependencies
- nltk *
- pandas *
- py-rouge ==1.1
- torch ==1.13.1
- transformers ==4.36.0