opennmt-py
Open Source Neural Machine Translation and (Large) Language Models in PyTorch
Science Score: 36.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
○CITATION.cff file
-
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
○DOI references
-
○Academic publication links
-
✓Committers with academic emails
19 of 190 committers (10.0%) from academic institutions -
○Institutional organization owner
-
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (13.9%) to scientific vocabulary
Keywords
Keywords from Contributors
Repository
Open Source Neural Machine Translation and (Large) Language Models in PyTorch
Basic Info
- Host: GitHub
- Owner: OpenNMT
- License: mit
- Language: Python
- Default Branch: master
- Homepage: https://opennmt.net/
- Size: 307 MB
Statistics
- Stars: 6,921
- Watchers: 178
- Forks: 2,257
- Open Issues: 33
- Releases: 50
Topics
Metadata Files
README.md
Announcement: OpenNMT-py is no longer actively supported.
We started a new project Eole available on Github
It is a spin-off of OpenNMT-py in terms of features but we revamped a lot of stuff.
Eole handles NMT, LLM, Encoders as well as a new concept of Estimator within a NMT Model See this post and this news
If you are a developer, switch now. If you are a user only, then we will publish the first py-pi versions shortly.
OpenNMT-py: Open-Source Neural Machine Translation and (Large) Language Models
OpenNMT-py is the PyTorch version of the OpenNMT project, an open-source (MIT) neural machine translation (and beyond!) framework. It is designed to be research friendly to try out new ideas in translation, language modeling, summarization, and many other NLP tasks. Some companies have proven the code to be production ready.
We love contributions! Please look at issues marked with the contributions welcome tag.
Before raising an issue, make sure you read the requirements and the Full Documentation examples.
Unless there is a bug, please use the Forum or Gitter to ask questions.
For beginners:
There is a step-by-step and explained tuto (Thanks to Yasmin Moslem): Tutorial
Please try to read and/or follow before raising newbies issues.
Otherwise you can just have a look at the Quickstart steps
New:
- You will need Pytorch v2 preferably v2.2 which fixes some
scaled_dot_product_attentionissues - LLM support with converters for: Llama (+ Mistral), OpenLlama, Redpajama, MPT-7B, Falcon.
- Support for 8bit and 4bit quantization along with LoRA adapters, with or without checkpointing.
- You can finetune 7B and 13B models on a single RTX 24GB with 4-bit quantization.
- Inference can be forced in 4/8bit using the same layer quantization as in finetuning.
- Tensor parallelism when the model does not fit on one GPU's memory (both training and inference)
- Once your model is finetuned you can run inference either with OpenNMT-py or faster with CTranslate2.
- MMLU evaluation script, see results here
For all usecases including NMT, you can now use Multiquery instead of Multihead attention (faster at training and inference) and remove biases from all Linear (QKV as well as FeedForward modules).
If you used previous versions of OpenNMT-py, you can check the Changelog or the Breaking Changes
Tutorials:
- How to replicate Vicuna with a 7B or 13B llama (or Open llama, MPT-7B, Redpajama) Language Model: Tuto Vicuna
- How to finetune NLLB-200 with your dataset: Tuto Finetune NLLB-200
- How to create a simple OpenNMT-py REST Server: Tuto REST
- How to create a simple Web Interface: Tuto Streamlit
- Replicate the WMT17 en-de experiment: WMT17 ENDE
Setup
Using docker
To facilitate setup and reproducibility, some docker images are made available via the Github Container Registry: https://github.com/OpenNMT/OpenNMT-py/pkgs/container/opennmt-py
You can adapt the workflow and build your own image(s) depending on specific needs by using build.sh and Dockerfile in the docker directory of the repo.
docker pull ghcr.io/opennmt/opennmt-py:3.4.3-ubuntu22.04-cuda12.1
Example oneliner to run a container and open a bash shell within it
docker run --rm -it --runtime=nvidia ghcr.io/opennmt/opennmt-py:test-ubuntu22.04-cuda12.1
Note: you need to have the Nvidia Container Toolkit (formerly nvidia-docker) installed to properly take advantage of the CUDA/GPU features.
Depending on your needs you can add various flags:
- -p 5000:5000 to forward some exposed port from your container to your host;
- -v /some/local/directory:/some/container/directory to mount some local directory to some container directory;
- --entrypoint some_command to directly run some specific command as the container entry point (instead of the default bash shell);
Installing locally
OpenNMT-py requires:
- Python >= 3.8
- PyTorch >= 2.0 <2.2
Install OpenNMT-py from pip:
bash
pip install OpenNMT-py
or from the source:
bash
git clone https://github.com/OpenNMT/OpenNMT-py.git
cd OpenNMT-py
pip install -e .
Note: if you encounter a MemoryError during installation, try to use pip with --no-cache-dir.
(Optional) Some advanced features (e.g. working pretrained models or specific transforms) require extra packages, you can install them with:
bash
pip install -r requirements.opt.txt
Manual installation of some dependencies
Apex is highly recommended to have fast performance (especially the legacy fusedadam optimizer and FusedRMSNorm)
shell
git clone https://github.com/NVIDIA/apex
cd apex
pip3 install -v --no-build-isolation --config-settings --build-option="--cpp_ext --cuda_ext --deprecated_fused_adam --xentropy --fast_multihead_attn" ./
cd ..
Flash attention:
As of Oct. 2023 flash attention 1 has been upstreamed to pytorch v2 but it is recommended to use flash attention 2 with v2.3.1 for sliding window attention support.
When using regular position_encoding=True or Rotary with max_relative_positions=-1 OpenNMT-py will try to use an optimized dot-product path.
if you want to use flash attention then you need to manually install it first:
bash
pip install flash-attn --no-build-isolation
if flash attention 2 is not installed, then we will use F.scaled_dot_product_attention from pytorch 2.x
When using max_relative_positions > 0 or Alibi max_relative_positions=-2 OpenNMT-py will use its legacy code for matrix multiplications.
flash attention and F.scaled_dot_product_attention are a bit faster and saves some GPU memory.
AWQ:
If you want to run inference or quantize an AWQ model you will need AutoAWQ.
For AutoAWQ: pip install autoawq
Documentation & FAQs
Acknowledgements
OpenNMT-py is run as a collaborative open-source project. Project was incubated by Systran and Harvard NLP in 2016 in Lua and ported to Pytorch in 2017.
Current maintainers (since 2018):
François Hernandez Vincent Nguyen (Seedfall)
Citation
If you are using OpenNMT-py for academic work, please cite the initial system demonstration paper published in ACL 2017:
@misc{klein2018opennmt,
title={OpenNMT: Neural Machine Translation Toolkit},
author={Guillaume Klein and Yoon Kim and Yuntian Deng and Vincent Nguyen and Jean Senellart and Alexander M. Rush},
year={2018},
eprint={1805.11462},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
Owner
- Name: OpenNMT
- Login: OpenNMT
- Kind: organization
- Website: https://opennmt.net/
- Repositories: 13
- Profile: https://github.com/OpenNMT
Open source ecosystem for neural machine translation and neural sequence learning
GitHub Events
Total
- Issues event: 3
- Watch event: 196
- Delete event: 1
- Issue comment event: 4
- Pull request event: 3
- Fork event: 38
- Create event: 2
Last Year
- Issues event: 3
- Watch event: 196
- Delete event: 1
- Issue comment event: 4
- Pull request event: 3
- Fork event: 38
- Create event: 2
Committers
Last synced: 9 months ago
Top Committers
| Name | Commits | |
|---|---|---|
| Vincent Nguyen | v****s@y****m | 376 |
| Sasha Rush | s****h@s****u | 284 |
| Benjamin Peters | b****s@c****e | 190 |
| Jianyu Zhan | n****6@g****m | 146 |
| pltrdy | p****y@g****m | 141 |
| François Hernandez | f****h@g****m | 121 |
| Sebastian Gehrmann | s****n@o****m | 117 |
| Guillaume Klein | g****n | 89 |
| Bryan Marcus McCann | b****s@g****m | 84 |
| yuntian | d****n@g****m | 70 |
| Adam Lerer | a****r@f****m | 40 |
| Linxiao ZENG | l****g@g****m | 39 |
| soumith | s****h@f****m | 38 |
| Lina Khodja | 5****5 | 35 |
| flauted | d****e@g****m | 34 |
| Jianri Li | s****r@p****u | 30 |
| Alexander Rush | s****h@s****u | 30 |
| Alexander Rush | s****h@s****u | 26 |
| wjbianjason | w****n@1****m | 19 |
| Hoang Viet | v****e@g****m | 16 |
| Ubuntu | u****u@L****t | 15 |
| Stig-Arne Gronroos | s****s@a****i | 13 |
| Justin Chiu | j****u@g****u | 13 |
| Adam Paszke | a****e@g****m | 12 |
| anderleich | a****5@g****m | 11 |
| Valentin Berkes | 1****0 | 11 |
| Gideon Wenniger | g****r@d****m | 10 |
| Panos Kanavos | p****s@g****m | 10 |
| ScarletPan | m****t@s****m | 9 |
| tayciryahmed | t****d@g****m | 8 |
| and 160 more... | ||
Committer Domains (Top 20 + Academic)
Issues and Pull Requests
Last synced: 6 months ago
All Time
- Total issues: 116
- Total pull requests: 225
- Average time to close issues: about 1 month
- Average time to close pull requests: 17 days
- Total issue authors: 86
- Total pull request authors: 26
- Average comments per issue: 2.9
- Average comments per pull request: 0.69
- Merged pull requests: 189
- Bot issues: 0
- Bot pull requests: 6
Past Year
- Issues: 4
- Pull requests: 4
- Average time to close issues: N/A
- Average time to close pull requests: about 2 months
- Issue authors: 4
- Pull request authors: 1
- Average comments per issue: 0.5
- Average comments per pull request: 0.5
- Merged pull requests: 0
- Bot issues: 0
- Bot pull requests: 4
Top Authors
Issue Authors
- vince62s (7)
- robertBrnnn (6)
- PC91 (4)
- HURIMOZ (3)
- anderleich (3)
- medicine-wave (3)
- Keram-Yasin (2)
- ILG2021 (2)
- Garfounkel (2)
- l-k-11235 (2)
- JOHW85 (2)
- Galaxy-Husky (2)
- fdggdfgg (2)
- yash-srivastava19 (2)
- timon49 (1)
Pull Request Authors
- vince62s (132)
- l-k-11235 (41)
- panosk (9)
- dependabot[bot] (7)
- anderleich (6)
- Ehsan-Jahanbakhsh (5)
- PC91 (5)
- robertBrnnn (4)
- funboarder13920 (3)
- alexis-allemann (3)
- Zenglinxiao (2)
- medfreeman (2)
- aaaallleen (2)
- stellarway (2)
- royshil (2)
Top Labels
Issue Labels
Pull Request Labels
Packages
- Total packages: 4
-
Total downloads:
- pypi 35,150 last-month
-
Total dependent packages: 3
(may contain duplicates) -
Total dependent repositories: 12
(may contain duplicates) - Total versions: 69
- Total maintainers: 4
pypi.org: opennmt-py
A python implementation of OpenNMT
- Documentation: http://opennmt.net/OpenNMT-py/
- License: mit
-
Latest release: 3.5.1
published almost 2 years ago
Rankings
Maintainers (3)
proxy.golang.org: github.com/opennmt/opennmt-py
- Documentation: https://pkg.go.dev/github.com/opennmt/opennmt-py#section-documentation
- License: mit
-
Latest release: v3.5.1+incompatible
published almost 2 years ago
Rankings
proxy.golang.org: github.com/OpenNMT/OpenNMT-py
- Documentation: https://pkg.go.dev/github.com/OpenNMT/OpenNMT-py#section-documentation
- License: mit
-
Latest release: v3.5.1+incompatible
published almost 2 years ago
Rankings
pypi.org: opennmt-py-svx
A python implementation of OpenNMT - Modified by Sonvx by adding a write_out flag
- Documentation: http://opennmt.net/OpenNMT-py/
- License: mit
-
Latest release: 1.1.1
published about 5 years ago