optimum

🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools

https://github.com/huggingface/optimum

Science Score: 36.0%

This score indicates how likely this project is to be science-related based on various indicators:

○
CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
○
DOI references
○
Academic publication links
✓
Committers with academic emails
5 of 145 committers (3.4%) from academic institutions
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (11.2%) to scientific vocabulary

Keywords

graphcore habana inference intel onnx onnxruntime optimization pytorch quantization tflite training transformers

Keywords from Contributors

transformer vlm pretrained-models speech-recognition qwen pytorch-transformers model-hub glm gemma deepseek

Last synced: 6 months ago · JSON representation

Repository

🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools

Basic Info

Host: GitHub
Owner: huggingface
License: apache-2.0
Language: Python
Default Branch: main
Homepage: https://huggingface.co/docs/optimum/main/
Size: 5.48 MB

Statistics

Stars: 3,056
Watchers: 57
Forks: 580
Open Issues: 280
Releases: 75

Topics

graphcore habana inference intel onnx onnxruntime optimization pytorch quantization tflite training transformers

Created over 4 years ago · Last pushed 6 months ago

Metadata Files

Readme Contributing License Code of conduct

🤗 Optimum

Optimum is an extension of Transformers 🤖 Diffusers 🧨 TIMM 🖼️ and Sentence-Transformers 🤗, providing a set of optimization tools and enabling maximum efficiency to train and run models on targeted hardware, while keeping things easy to use.

Installation

Optimum can be installed using pip as follows:

bash python -m pip install optimum

If you'd like to use the accelerator-specific features of Optimum, you can check the documentation and install the required dependencies according to the table below:

| Accelerator | Installation | | :---------------------------------------------------------------------------------- | :-------------------------------------------------------------------------- | | ONNX | pip install --upgrade --upgrade-strategy eager optimum[onnx] | | Intel Neural Compressor | pip install --upgrade --upgrade-strategy eager optimum[neural-compressor] | | OpenVINO | pip install --upgrade --upgrade-strategy eager optimum[openvino] | | IPEX | pip install --upgrade --upgrade-strategy eager optimum[ipex] | | NVIDIA TensorRT-LLM | docker run -it --gpus all --ipc host huggingface/optimum-nvidia | | AMD Instinct GPUs and Ryzen AI NPU | pip install --upgrade --upgrade-strategy eager optimum[amd] | | AWS Trainum & Inferentia | pip install --upgrade --upgrade-strategy eager optimum[neuronx] | | Intel Gaudi Accelerators (HPU) | pip install --upgrade --upgrade-strategy eager optimum[habana] | | FuriosaAI | pip install --upgrade --upgrade-strategy eager optimum[furiosa] |

The --upgrade --upgrade-strategy eager option is needed to ensure the different packages are upgraded to the latest possible version.

To install from source:

bash python -m pip install git+https://github.com/huggingface/optimum.git

For the accelerator-specific features, append optimum[accelerator_type] to the above command:

bash python -m pip install optimum[onnxruntime]@git+https://github.com/huggingface/optimum.git

Accelerated Inference

Optimum provides multiple tools to export and run optimized models on various ecosystems:

ONNX / ONNX Runtime, one of the most popular open formats for model export, and a high-performance inference engine for deployment.
OpenVINO, a toolkit for optimizing, quantizing and deploying deep learning models on Intel hardware.
ExecuTorch, PyTorch’s native solution for on-device inference across mobile and edge devices.
Intel Gaudi Accelerators enabling optimal performance on first-gen Gaudi, Gaudi2 and Gaudi3.
AWS Inferentia for accelerated inference on Inf2 and Inf1 instances.
NVIDIA TensorRT-LLM.

The export and optimizations can be done both programmatically and with a command line.

ONNX + ONNX Runtime

🚨🚨🚨 ONNX integration moving to optimum-onnx so make sure to follow the installation instructions 🚨🚨🚨

Before you begin, make sure you have all the necessary libraries installed :

bash pip install --upgrade --upgrade-strategy eager optimum[onnx]

It is possible to export Transformers, Diffusers, Sentence Transformers and timm models to the ONNX format and perform graph optimization as well as quantization easily.

For more information on the ONNX export, please check the documentation.

Once the model is exported to the ONNX format, we provide Python classes enabling you to run the exported ONNX model in a seemless manner using ONNX Runtime in the backend.

For this make sure you have ONNX Runtime installed, fore more information check out the installation instructions.

More details on how to run ONNX models with ORTModelForXXX classes here.

Intel (OpenVINO + Neural Compressor + IPEX)

Before you begin, make sure you have all the necessary libraries installed.

You can find more information on the different integration in our documentation and in the examples of optimum-intel.

ExecuTorch

Before you begin, make sure you have all the necessary libraries installed :

bash pip install optimum-executorch@git+https://github.com/huggingface/optimum-executorch.git

Users can export Transformers models to ExecuTorch and run inference on edge devices within PyTorch's ecosystem.

For more information about export Transformers to ExecuTorch, please check the doc for Optimum-ExecuTorch.

Quanto

Quanto is a pytorch quantization backend which allows you to quantize a model either using the python API or the optimum-cli.

You can see more details and examples in the Quanto repository.

Accelerated training

Optimum provides wrappers around the original Transformers Trainer to enable training on powerful hardware easily. We support many providers:

Intel Gaudi Accelerators (HPU) enabling optimal performance on first-gen Gaudi, Gaudi2 and Gaudi3.
AWS Trainium for accelerated training on Trn1 and Trn1n instances.
ONNX Runtime (optimized for GPUs).

Intel Gaudi Accelerators

Before you begin, make sure you have all the necessary libraries installed :

bash pip install --upgrade --upgrade-strategy eager optimum[habana]

You can find examples in the documentation and in the examples.

AWS Trainium

Before you begin, make sure you have all the necessary libraries installed :

bash pip install --upgrade --upgrade-strategy eager optimum[neuronx]

You can find examples in the documentation and in the tutorials.

ONNX Runtime

Before you begin, make sure you have all the necessary libraries installed :

bash pip install optimum[onnxruntime-training]

You can find examples in the documentation and in the examples.

Owner

Name: Hugging Face
Login: huggingface
Kind: organization
Location: NYC + Paris

Website: https://huggingface.co/
Twitter: huggingface
Repositories: 344
Profile: https://github.com/huggingface

The AI community building the future.

Committers

Last synced: 9 months ago

All Time

Total Commits: 1,223
Total Committers: 145
Avg Commits per committer: 8.434
Development Distribution Score (DDS): 0.725

Past Year

Commits: 188
Committers: 48
Avg Commits per committer: 3.917
Development Distribution Score (DDS): 0.644

Top Committers

Name	Email	Commits
fxmarty	9****y	336
Ella Charlaix	8****x	268
Jingya HUANG	4****g	97
regisss	1****s	73
Michael Benayoun	m**n@g**m	53
Ilyas Moutawwakil	5****l	43
Mohit Sharma	m**s@g**m	34
lewtun	l**l@g**m	22
Younes Belkada	4****a	20
François Lagunas	f**s@g**m	20
Marc Sun	5****c	16
Funtowicz Morgan	m****z	16
Joshua Lochner	a**n@x**m	15
Bas Krahmer	b**r@g**m	12
Philipp Schmid	3****d	10
Ekaterina Aidova	e**a@i**m	9
Adam Louly	a**3@g**m	8
Prathik Rao	p**o@g**m	8
Mishig	m**j@c**u	6
Longjie Zheng	3****x	4
Wang, Chang	c**g@i**m	4
jiqing-feng	j**g@i**m	3
BADAOUI Abdennacer	1****i	3
David Corvoysier	d**r@g**m	3
Ryan Russell	r****l	3
Tom Aarsen	3****n	3
jingyanwangms	4****s	3
kunal-vaishnavi	1****i	3
LRL-ModelCloud	1****d	2
Luc Georges	M****e	2
and 115 more...

Committer Domains (Top 20 + Academic)

intel.com: 9 buaa.edu.cn: 2 xenova.com: 1 coloradocollege.edu: 1 botpress.com: 1 uniartisan.com: 1 hans00.me: 1 microsoft.com: 1 naver.com: 1 eurecom.fr: 1 buster.ai: 1 ugent.be: 1 alibaba-inc.com: 1 163.com: 1 hey.com: 1 vikas.sh: 1 126.com: 1 umich.edu: 1 moveworks.ai: 1 jqlf.fr: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time

Total issues: 559
Total pull requests: 1,019
Average time to close issues: 6 months
Average time to close pull requests: about 2 months
Total issue authors: 432
Total pull request authors: 172
Average comments per issue: 2.13
Average comments per pull request: 1.85
Merged pull requests: 637
Bot issues: 0
Bot pull requests: 1

Past Year

Issues: 113
Pull requests: 348
Average time to close issues: 28 days
Average time to close pull requests: 19 days
Issue authors: 99
Pull request authors: 59
Average comments per issue: 0.77
Average comments per pull request: 1.72
Merged pull requests: 213
Bot issues: 0
Bot pull requests: 1

View more stats

Top Authors

Issue Authors

fxmarty (25)
xenova (15)
gidzr (8)
IlyasMoutawwakil (6)
michaelbenayoun (6)
AdamLouly (5)
JingyaHuang (4)
Harini-Vemula-2382 (4)
prathikr (4)
pradeepdev-1995 (4)
michaelroyzen (4)
tomaarsen (3)
Kaya-P (3)
jyangliu (3)
ZeusFSX (3)

Pull Request Authors

echarlaix (211)
fxmarty (149)
IlyasMoutawwakil (86)
xenova (53)
JingyaHuang (36)
regisss (36)
mht-sharma (28)
eaidova (20)
SunMarc (17)
baskrahmer (17)
jiqing-feng (12)
Abdennacer-Badaoui (12)
zhenglongjiepheonix (10)
changwangss (10)
michaelbenayoun (9)

Top Labels

Issue Labels

bug (271) Stale (79) feature-request (55) onnx (47) onnxruntime (26) bettertransformer (15) exporters (4) quantization (4) question (4) good first issue (3) tflite (2) documentation (2) training (2) help wanted (1) graph-optimization (1) inference (1)

Pull Request Labels

Stale (104) build-pr-doc (11) onnxruntime-slow (8) gpu-test (6) training (6) bug (3) onnxruntime-gpu (3) onnxruntime-training (3) onnx (2) tflite (2) dependencies (1) python (1) onnxruntime (1)

Packages

Total packages: 4
Total downloads:
- pypi 1,337,008 last-month
Total docker downloads: 14,124

Total dependent packages: 84
(may contain duplicates)
Total dependent repositories: 715
(may contain duplicates)
Total versions: 173
Total maintainers: 7

pypi.org: optimum

Optimum Library is an extension of the Hugging Face Transformers library, providing a framework to integrate third-party libraries from Hardware Partners and interface with their specific functionality.

Homepage: https://github.com/huggingface/optimum
Documentation: https://optimum.readthedocs.io/
License: Apache
Latest release: 1.27.0
published 7 months ago

Versions: 82
Dependent Packages: 82
Dependent Repositories: 715
Downloads: 1,337,008 Last month
Docker Downloads: 14,124

Rankings

Dependent packages count: 0.3%

Downloads: 0.4%

Dependent repos count: 0.5%

Average: 1.4%

Stargazers count: 1.7%

Docker downloads count: 2.1%

Forks count: 3.2%

Maintainers (7)

regisss echarlaix michaelbenayoun fxmarty mfuntowicz Jhuaplin david_hf

Last synced: 6 months ago

proxy.golang.org: github.com/huggingface/optimum

Documentation: https://pkg.go.dev/github.com/huggingface/optimum#section-documentation
License: apache-2.0
Latest release: v1.27.0
published 7 months ago

Versions: 80
Dependent Packages: 0
Dependent Repositories: 0

Rankings

Dependent packages count: 5.5%

Average: 5.7%

Dependent repos count: 5.9%

Last synced: 6 months ago

conda-forge.org: optimum

🤗 Optimum is an extension of 🤗 Transformers, providing a set of performance optimization tools enabling maximum efficiency to train and run models on targeted hardware. The AI ecosystem evolves quickly and more and more specialized hardware along with their own optimizations are emerging every day. As such, Optimum enables users to efficiently use any of these platforms with the same ease inherent to transformers. PyPI: [https://pypi.org/project/optimum/](https://pypi.org/project/optimum/)

Homepage: https://huggingface.co/hardware
License: Apache-2.0
Latest release: 1.3.0
published over 3 years ago

Versions: 7
Dependent Packages: 2
Dependent Repositories: 0

Rankings

Stargazers count: 13.3%

Forks count: 19.1%

Dependent packages count: 19.5%

Average: 21.5%

Dependent repos count: 34.0%

Last synced: 6 months ago

anaconda.org: optimum

🤗 Optimum is an extension of Transformers that provides a set of performance optimization tools to train and run models on targeted hardware with maximum efficiency. The AI ecosystem evolves quickly, and more and more specialized hardware along with their own optimizations are emerging every day. As such, Optimum enables developers to efficiently use any of these platforms with the same ease inherent to Transformers.

Homepage: https://github.com/huggingface/optimum
License: Apache-2.0
Latest release: 1.24.0
published 12 months ago

Versions: 4
Dependent Packages: 0
Dependent Repositories: 0

Rankings

Stargazers count: 24.2%

Forks count: 31.3%

Average: 38.2%

Dependent packages count: 39.8%

Dependent repos count: 57.7%

Last synced: 6 months ago

Dependencies

.github/workflows/build_main_documentation.yml actions

actions/checkout v2 composite

.github/workflows/build_pr_documentation.yml actions

actions/checkout v2 composite
actions/upload-artifact v3 composite

.github/workflows/check_code_quality.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/dev_test_benckmark.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/dev_test_bettertransformer.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/dev_test_dummy_inputs.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/dev_test_exporters.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/dev_test_fx.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/dev_test_onnx.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/dev_test_onnxruntime.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/dev_test_optimum_common.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/doctests.yml actions

actions/checkout v2 composite
aws-actions/configure-aws-credentials v1 composite
philschmid/philschmid-ec2-github-runner main composite

.github/workflows/test_benckmark.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/test_bettertransformer.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/test_bettertransformer_gpu.yml actions

actions/checkout v2 composite
aws-actions/configure-aws-credentials v1 composite
philschmid/philschmid-ec2-github-runner main composite

.github/workflows/test_cli.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/test_dummy_inputs.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/test_export_onnx.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/test_export_onnx_cli.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/test_export_onnx_cli_timm.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/test_export_onnx_timm.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/test_export_tflite.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/test_export_tflite_cli.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/test_export_tflite_cli_dynamic_quantization_int8.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/test_export_tflite_cli_quantization_fp16.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/test_export_tflite_cli_quantization_full_int8.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/test_export_tflite_cli_quantization_int8_custom_dataset.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/test_export_tflite_cli_quantization_int8_default_dataset.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/test_export_tflite_cli_quantization_int8x16.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/test_exporters_common.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/test_exporters_gpu.yml actions

actions/checkout v2 composite
aws-actions/configure-aws-credentials v1 composite
philschmid/philschmid-ec2-github-runner main composite

.github/workflows/test_exporters_slow.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/test_fx.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/test_gptq.yml actions

actions/checkout v2 composite
aws-actions/configure-aws-credentials v1 composite
philschmid/philschmid-ec2-github-runner main composite

.github/workflows/test_onnx.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/test_onnxruntime.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/test_onnxruntime_gpu.yml actions

actions/checkout v2 composite
aws-actions/configure-aws-credentials v1 composite
philschmid/philschmid-ec2-github-runner main composite

.github/workflows/test_onnxruntime_train.yml actions

actions/checkout v2 composite
aws-actions/configure-aws-credentials v1 composite
philschmid/philschmid-ec2-github-runner main composite

.github/workflows/test_optimum_common.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

.github/workflows/upload_pr_documentation.yml actions

docs/Dockerfile docker

nikolaik/python-nodejs python3.8-nodejs18 build

examples/onnxruntime/optimization/multiple-choice/requirements.txt pypi

datasets >=1.8.0
onnx *
onnxruntime >=1.9.0
protobuf *
scikit-learn *
scipy *
sentencepiece *
torch >=1.9

examples/onnxruntime/optimization/question-answering/requirements.txt pypi

datasets >=1.8.0
onnx *
onnxruntime >=1.9.0
torch >=1.9.0

examples/onnxruntime/optimization/text-classification/requirements.txt pypi

datasets >=1.8.0
onnx *
onnxruntime >=1.9.0
protobuf *
scikit-learn *
scipy *
sentencepiece *
torch >=1.9

examples/onnxruntime/optimization/token-classification/requirements.txt pypi

datasets >=1.18.0
onnx *
onnxruntime >=1.9.0
seqeval *
torch >=1.9

examples/onnxruntime/quantization/image-classification/requirements.txt pypi

datasets >=1.17.0
torch >=1.5.0
torchvision >=0.6.0

examples/onnxruntime/quantization/multiple-choice/requirements.txt pypi

datasets >=1.8.0
onnx *
onnxruntime >=1.9.0
protobuf *
scikit-learn *
scipy *
sentencepiece *
torch >=1.9

examples/onnxruntime/quantization/question-answering/requirements.txt pypi

datasets >=1.8.0
onnx *
onnxruntime >=1.9.0
torch >=1.9.0

examples/onnxruntime/quantization/text-classification/requirements.txt pypi

datasets >=1.8.0
onnx *
onnxruntime >=1.9.0
protobuf *
scikit-learn *
scipy *
sentencepiece *
torch >=1.9

examples/onnxruntime/quantization/token-classification/requirements.txt pypi

datasets >=1.8.0
onnx *
onnxruntime >=1.9.0
seqeval *
torch >=1.9

examples/onnxruntime/training/image-classification/requirements.txt pypi

accelerate >=0.12.0
datasets >=1.17.0
evaluate *
onnx >=1.9.0
onnxruntime-training >=1.9.0
torch >=1.5.0
torch-ort *
torchvision >=0.6.0

examples/onnxruntime/training/language-modeling/requirements.txt pypi

datasets >=1.8.0
onnx >=1.9.0
onnxruntime-training >=1.9.0
protobuf ==3.20.2
scikit-learn *
scipy *
sentencepiece *
torch >=1.9.0
torch-ort *
transformers >=4.16.0

examples/onnxruntime/training/question-answering/requirements.txt pypi

datasets >=1.8.0
protobuf *
scikit-learn *
scipy *
sentencepiece *
torch >=1.9.0
torch-ort *

examples/onnxruntime/training/stable-diffusion/text-to-image/requirements.txt pypi

Jinja2 *
accelerate >=0.16.0
datasets *
ftfy *
tensorboard *
transformers >=4.25.1

examples/onnxruntime/training/summarization/requirements.txt pypi

accelerate *
datasets >=1.8.0
evaluate *
nltk *
protobuf *
py7zr *
rouge-score *
scikit-learn *
scipy *
sentencepiece *
torch >=1.9.0
torch-ort *

examples/onnxruntime/training/text-classification/requirements.txt pypi

datasets >=1.8.0
protobuf *
scikit-learn *
scipy *
sentencepiece *

examples/onnxruntime/training/token-classification/requirements.txt pypi

datasets >=1.18.3
scikit-learn *
scipy *
sentencepiece *
seqeval *
torch >=1.8.1
torch >=1.9
torch-ort *

examples/onnxruntime/training/translation/requirements.txt pypi

datasets >=1.18.0
protobuf *
py7zr *
sacrebleu >=1.4.12
sentencepiece *
torch >=1.8

pyproject.toml pypi

setup.py pypi

.github/workflows/test_onnxruntime_slow.yml actions

actions/checkout v2 composite
actions/setup-python v2 composite

optimum

Science Score: 36.0%

Keywords

Keywords from Contributors

Repository

Basic Info

Statistics

Topics

Metadata Files

README.md

🤗 Optimum

Installation

Accelerated Inference

ONNX + ONNX Runtime

Intel (OpenVINO + Neural Compressor + IPEX)

ExecuTorch

Quanto

Accelerated training

Intel Gaudi Accelerators

AWS Trainium

ONNX Runtime

Owner

Committers

All Time

Past Year

Top Committers

Committer Domains (Top 20 + Academic)

Issues and Pull Requests

All Time

Past Year

Top Authors

Issue Authors

Pull Request Authors

Top Labels

Issue Labels

Pull Request Labels

Packages

pypi.org: optimum

Rankings

Maintainers (7)

proxy.golang.org: github.com/huggingface/optimum

Rankings

conda-forge.org: optimum

Rankings

anaconda.org: optimum

Rankings

Dependencies