nemo

NeMo (PyTorch TorchScript) models to LiteRT (formerly TFLite) Direct Conversion

https://github.com/peblove/nemo

Science Score: 44.0%

This score indicates how likely this project is to be science-related based on various indicators:

✓
CITATION.cff file
Found CITATION.cff file
✓
codemeta.json file
Found codemeta.json file
✓
.zenodo.json file
Found .zenodo.json file
○
DOI references
○
Academic publication links
○
Academic email domains
○
Institutional organization owner
○
JOSS paper metadata
○
Scientific vocabulary similarity
Low similarity (7.3%) to scientific vocabulary

Last synced: 10 months ago · JSON representation ·

Repository

NeMo (PyTorch TorchScript) models to LiteRT (formerly TFLite) Direct Conversion

Basic Info

Host: GitHub
Owner: peblove
License: apache-2.0
Language: Python
Default Branch: main
Homepage:
Size: 157 MB

Statistics

Stars: 1
Watchers: 1
Forks: 0
Open Issues: 0
Releases: 0

Created over 1 year ago · Last pushed over 1 year ago

Metadata Files

Readme Changelog Contributing License Citation Codeowners

README.md

NeMo (PyTorch TorchScript) models to LiteRT (formerly TFLite) Direct Conversion

This repository demonstrates how to directly convert NeMo models to LiteRT (formerly TFLite) int8 quantized models without using ONNX as an intermediate step.

Environment Prerequisites

Due to compatibility requirements: - Python 3.10 is recommended (aiedgetorch does not currently support Python 3.12) - CUDA-compatible environment

Installation

1. System Dependencies

Install required system packages: bash apt-get update && apt-get install -y libsndfile1 ffmpeg

2. TensorFlow Installation

Install TensorFlow with CUDA support first due to version compatibility with PyTorch 2.5.1: bash pip install tensorflow[and-cuda]==2.18.0

3. Additional Dependencies

Install remaining required packages: bash pip install -r requirements.txt

Model Conversion

To convert the stt_en_fastconformer_ctc_large model to LiteRT format: bash python nemo2litert.py

Additional Features

Supports conversion of cache-aware streaming models
Compatible with dynamic axes models
Enables direct int8 quantization

Technical Notes

The conversion process bypasses ONNX intermediates for more efficient transformation
Supports various NeMo model architectures
Maintains model performance while reducing size through int8 quantization

Owner

Name: Hyunsik Kim
Login: peblove
Kind: user
Company: home

Repositories: 1
Profile: https://github.com/peblove

AI/ML Researcher with expertise in Speech Recognition and ARM Architecture

Citation (CITATION.cff)

cff-version: 1.2.0
message: "If you use this software, please cite it as below."
title: "NeMo: a toolkit for Conversational AI and Large Language Models"
url: https://nvidia.github.io/NeMo/
repository-code: https://github.com/NVIDIA/NeMo
authors:
  - family-names: Harper
    given-names: Eric
  - family-names: Majumdar
    given-names: Somshubra
  - family-names: Kuchaiev
    given-names: Oleksii
  - family-names: Jason
    given-names: Li
  - family-names: Zhang
    given-names: Yang
  - family-names: Bakhturina
    given-names: Evelina
  - family-names: Noroozi 
    given-names: Vahid
  - family-names: Subramanian
    given-names: Sandeep
  - family-names: Nithin
    given-names: Koluguri
  - family-names: Jocelyn
    given-names: Huang
  - family-names: Jia
    given-names: Fei
  - family-names: Balam
    given-names: Jagadeesh
  - family-names: Yang
    given-names: Xuesong
  - family-names: Livne
    given-names: Micha
  - family-names: Dong
    given-names: Yi
  - family-names: Naren
    given-names: Sean
  - family-names: Ginsburg
    given-names: Boris

GitHub Events

Total

Issues event: 2
Watch event: 3
Issue comment event: 11
Push event: 8
Create event: 2

Last Year

Issues event: 2
Watch event: 3
Issue comment event: 11
Push event: 8
Create event: 2

Issues and Pull Requests

Last synced: 10 months ago

All Time

Total issues: 1
Total pull requests: 0
Average time to close issues: 3 days
Average time to close pull requests: N/A
Total issue authors: 1
Total pull request authors: 0
Average comments per issue: 1.0
Average comments per pull request: 0
Merged pull requests: 0
Bot issues: 0
Bot pull requests: 0

Past Year

Issues: 1
Pull requests: 0
Average time to close issues: 3 days
Average time to close pull requests: N/A
Issue authors: 1
Pull request authors: 0
Average comments per issue: 1.0
Average comments per pull request: 0
Merged pull requests: 0
Bot issues: 0
Bot pull requests: 0

View more stats

Top Authors

Issue Authors

AymaneD2000 (1)

Pull Request Authors

Top Labels

Issue Labels

bug (1)

Pull Request Labels

Dependencies

.github/actions/cancel-workflow/action.yml actions

.github/workflows/_test_template.yml actions

NVIDIA/NeMo/.github/actions/cancel-workflow main composite

.github/workflows/build-test-publish-wheel.yml actions

.github/workflows/changelog-build.yml actions

actions/checkout v4 composite
mikepenz/release-changelog-builder-action v3.3.1 composite
peter-evans/create-pull-request v7 composite

.github/workflows/cherry-pick-release-commit.yml actions

.github/workflows/cicd-main.yml actions

peter-evans/create-or-update-comment v4 composite

.github/workflows/close-inactive-issue-pr.yml actions

actions/stale v6 composite

.github/workflows/code-formatting.yml actions

EndBug/add-and-commit v9 composite
actions/checkout v4 composite
actions/setup-python v5 composite
isort/isort-action v1 composite
peter-evans/create-or-update-comment v4 composite
peter-evans/find-comment v3 composite
psf/black stable composite
tj-actions/changed-files v44 composite

.github/workflows/codeql.yml actions

actions/checkout v3 composite
github/codeql-action/analyze v2 composite
github/codeql-action/autobuild v2 composite
github/codeql-action/init v2 composite

.github/workflows/config/codeql.yml actions

.github/workflows/copyright-check.yml actions

.github/workflows/gh-docs.yml actions

actions/checkout v3 composite

.github/workflows/import-test.yml actions

actions/checkout v2 composite
actions/setup-python v5 composite

.github/workflows/labeler.yml actions

actions/labeler v4 composite

.github/workflows/mcore-tag-bump-bot.yml actions

.github/workflows/monitor-single-vm.yml actions

.github/workflows/monitor-vms.yml actions

.github/workflows/release-freeze.yml actions

actions/checkout v4 composite
peter-evans/create-pull-request v6 composite

.github/workflows/release.yml actions

.github/workflows/secrets-detector.yml actions

EndBug/add-and-commit v9 composite
actions/checkout v4 composite

pyproject.toml pypi

requirements/requirements.txt pypi

huggingface_hub >=0.24
numba *
numpy >=1.22
onnx >=1.7.0
protobuf ==3.20.3
python-dateutil *
ruamel.yaml *
scikit-learn *
setuptools >=70.0.0
tensorboard *
text-unidecode *
torch *
tqdm >=4.41.0
wget *
wrapt *

requirements/requirements_asr.txt pypi

braceexpand *
editdistance *
einops *
g2p_en *
jiwer *
kaldi-python-io *
kaldiio *
lhotse >=1.26.0
librosa >=0.10.2
marshmallow *
optuna *
packaging *
pyannote.core *
pyannote.metrics *
pydub *
pyloudnorm *
resampy *
ruamel.yaml *
scipy >=0.14
soundfile *
sox *
texterrors *

requirements/requirements_audio.txt pypi

einops *
lhotse >=1.22.0
librosa >=0.10.0
matplotlib *
pesq *
pystoi *
scipy >=0.14
soundfile *
sox *

requirements/requirements_common.txt pypi

datasets *
einops *
inflect *
mediapy ==1.1.6
pandas *
sacremoses >=0.0.43
sentencepiece <1.0.0

requirements/requirements_deploy.txt pypi

fastapi *
nvidia-pytriton *
pydantic-settings *
tensorstore *
uvicorn *
zarr *

requirements/requirements_docs.txt pypi

Jinja2 *
Sphinx *
boto3 *
latexcodec *
numpy *
pydata-sphinx-theme *
sphinx-book-theme *
sphinx-copybutton *
sphinxcontrib-bibtex *
sphinxext-opengraph *
urllib3 *
wrapt *

requirements/requirements_eval.txt pypi

lm-eval *

requirements/requirements_infer.txt pypi

fastapi *
nvidia-pytriton *
pydantic-settings *
tensorstore *
uvicorn *
zarr *

requirements/requirements_lightning.txt pypi

cloudpickle *
fiddle *
hydra-core >1.3,<=1.3.2
lightning >2.2.1,<=2.4.0
omegaconf <=2.3
peft *
torchmetrics >=0.11.0
transformers >=4.45.0
wandb *
webdataset >=0.2.86

requirements/requirements_multimodal.txt pypi

PyMCubes *
addict *
bitsandbytes ==0.45.0
clip *
decord *
diffusers >=0.19.3
einops_exts *
imageio *
kornia *
megatron-energon ==4.0.0
nerfacc >=0.5.3
open_clip_torch ==2.24.0
taming-transformers *
torchdiffeq *
torchsde *
trimesh *

requirements/requirements_nlp.txt pypi

accelerated-scan *
boto3 *
faiss-cpu *
fasttext *
flask_restful *
ftfy *
gdown *
h5py *
ijson *
jieba *
markdown2 *
matplotlib >=3.3.2
nltk >=3.6.5
numpy <2
opencc *
pangu *
prettytable *
rapidfuzz *
rouge_score *
sacrebleu *
sentence_transformers *
tensorstore *
tiktoken ==0.7.0
zarr *

requirements/requirements_slu.txt pypi

jiwer >=2.0.0
progress >=1.5
tabulate >=0.8.7
textdistance >=4.1.5
tqdm *

requirements/requirements_test.txt pypi

black * test
click >=8.1 test
isort >5.1.0,<6.0.0 test
parameterized * test
pytest * test
pytest-mock * test
pytest-runner * test
ruamel.yaml * test
sphinx * test
sphinxcontrib-bibtex * test
wandb * test
wget * test
wrapt * test

requirements/requirements_tts.txt pypi

attrdict *
cdifflib ==1.2.6
einops *
janome *
jieba *
kornia *
librosa *
matplotlib *
nemo_text_processing *
nltk *
pandas *
pypinyin *
pypinyin-dict *
seaborn *

requirements/requirements_vllm.txt pypi

OpenCC *
braceexpand *
faiss-cpu *
fiddle *
h5py *
hydra-core >1.3,<=1.3.2
ijson *
jieba *
lightning >2.2.1
matplotlib >=3.3.2
omegaconf <=2.3
onnx >=1.7.0
pangu *
rouge_score *
sacrebleu *
scikit-learn *
vllm ==0.6.3
webdataset >=0.2.86
wget *

requirements.txt pypi

Cython *
ai_edge_torch *
nemo_toolkit *
packaging *
torch ==2.5.1
torch_xla ==2.5.1

scripts/freesound_download_resample/freesound_requirements.txt pypi

joblib *
librosa *
requests *
requests_oauthlib *
sox *

setup.py pypi

tools/ctc_segmentation/requirements.txt pypi

ctc_segmentation ==1.7.1
num2words *

tools/nemo_forced_aligner/requirements.txt pypi

nemo_toolkit *
prettyprinter *
pytest *

tools/nmt_webapp/requirements.txt pypi

flask *
flask_cors *
nemo_toolkit >=1.0.0rc1

tools/speech_data_explorer/requirements.txt pypi

SoundFile *
dash >=2.1.0
dash_bootstrap_components >=1.0.3
diff_match_patch *
editdistance *
jiwer *
librosa >=0.9.1
numpy *
plotly *
sox *
tqdm *

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Open Source Science

nemo

Science Score: 44.0%

Repository

Basic Info

Statistics

Metadata Files

README.md

NeMo (PyTorch TorchScript) models to LiteRT (formerly TFLite) Direct Conversion

Environment Prerequisites

Installation

1. System Dependencies

2. TensorFlow Installation

3. Additional Dependencies

Model Conversion

Additional Features

Technical Notes

Owner

Citation (CITATION.cff)

GitHub Events

Total

Last Year

Issues and Pull Requests

All Time

Past Year

Top Authors

Issue Authors

Pull Request Authors

Top Labels

Issue Labels

Pull Request Labels

Dependencies