deepquest-py
Large and Distilled Models for Quality Estimation of Machine Translation
Science Score: 18.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
○codemeta.json file
-
○.zenodo.json file
-
○DOI references
-
○Academic publication links
-
○Academic email domains
-
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (11.7%) to scientific vocabulary
Keywords
Repository
Large and Distilled Models for Quality Estimation of Machine Translation
Basic Info
Statistics
- Stars: 7
- Watchers: 13
- Forks: 1
- Open Issues: 1
- Releases: 0
Topics
Metadata Files
README.md
deepQuest-py
deepQuest-py is a framework for training and evaluation of models for Quality Estimation of Machine Translation. This is a new version of deepQuest - the first framework for neural Quality Estimation.
deepQuest-py provides:
- high performing sentence-level and word-level models based on finetuning pre-trained Transformers;
- light-weight and efficient sentence-level models implemented via knowledge distillation.
deepQuest-py includes implementations of several approaches for Quality Estimation proposed in recent research:
- Knowledge Distillation for Quality Estimation (Gajbhiye et al., 2021)
- TransQuest at WMT2020: Sentence-Level Direct Assessment (Ranasinghe et al., 2020)
- Two-Phase Cross-Lingual Language Model Fine-Tuning for Machine Translation Quality Estimation (Lee, 2020)
- deepQuest: A Framework for Neural-based Quality Estimation (Ive et al., 2018)
See our examples for instructions on how to train and test specific models.
Online Demo
Check out our web tool to try out most of our trained models on your own data!
Installation
deepQuest-py requires Python 3.6 or later.
git clone https://github.com/sheffieldnlp/deepQuest-py.git
cd deepQuest-py
pip install -e .
Licence
deepQuest-py is licenced under a CC BY-NC-SA licence.
Citation
If you use deepQuest-py in your research, please cite our EMNLP 2021 Demo paper:
@inproceedings{alva-manchego-etal-2021-deepquest,
title = "deep{Q}uest-py: {L}arge and Distilled Models for Quality Estimation",
author = "Alva-Manchego, Fernando and
Obamuyide, Abiola and
Gajbhiye, Amit and
Blain, Fr{\'e}d{\'e}ric and
Fomicheva, Marina and
Specia, Lucia",
booktitle = "Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations",
month = nov,
year = "2021",
address = "Online and Punta Cana, Dominican Republic",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2021.emnlp-demo.42",
pages = "382--389",
}
Owner
- Name: University of Sheffield - Natural Language Processing
- Login: sheffieldnlp
- Kind: organization
- Location: Sheffield, UK
- Website: https://www.sheffield.ac.uk/dcs/research/groups/natural-language-processing
- Repositories: 15
- Profile: https://github.com/sheffieldnlp
Citation (CITATION.bib)
@inproceedings{alva-manchego-etal-2021-deepquest,
title = "deep{Q}uest-py: {L}arge and Distilled Models for Quality Estimation",
author = "Alva-Manchego, Fernando and
Obamuyide, Abiola and
Gajbhiye, Amit and
Blain, Fr{\'e}d{\'e}ric and
Fomicheva, Marina and
Specia, Lucia",
booktitle = "Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations",
month = nov,
year = "2021",
address = "Online and Punta Cana, Dominican Republic",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2021.emnlp-demo.42",
pages = "382--389",
abstract = "We introduce deepQuest-py, a framework for training and evaluation of large and light-weight models for Quality Estimation (QE). deepQuest-py provides access to (1) state-of-the-art models based on pre-trained Transformers for sentence-level and word-level QE; (2) light-weight and efficient sentence-level models implemented via knowledge distillation; and (3) a web interface for testing models and visualising their predictions. deepQuest-py is available at \url{https://github.com/sheffieldnlp/deepQuest-py} under a CC BY-NC-SA licence.",
}
GitHub Events
Total
Last Year
Dependencies
- allennlp ==2.1.0
- datasets *
- numpy *
- pandas *
- scikit-learn *
- scipy *
- tokenizers *
- tqdm *
- transformers >=4.8