This repository was maintained by dusty-nv, who stepped down on Monday. We thank them for their contributions.

CUDA Containers for Edge AI & Robotics

Modular container build system that provides the latest AI/ML packages for NVIDIA Jetson :rocket::robot:

[!WARNING] pypi.jetson-ai-lab.io is down. We are improving security and performance. Meanwhile please use pypi.jetson-ai-lab.io by setting the enviroment variable INDEX_HOST=jetson-ai-lab.io in .env file.

Only Tested and supported Jetpack 6.2 (Cuda 12.6) and JetPack 7 (CUDA 13.x).

[!NOTE] Ubuntu 24.04 containers for JetPack 6 and JetPack 7 are now available (with CUDA support)

     LSB_RELEASE=24.04 jetson-containers build pytorch:2.8
     jetson-containers run dustynv/pytorch:2.8-r36.4-cu128-24.04

ARM SBSA (Server Base System Architecture) is supported for GH200 / GB200.
To install CUDA 13.0 SBSA wheels for Python 3.12 / 24.04:

     pip3 install torch torchvision torchaudio \
            --index-url https://pypi.jetson-ai-lab.io/sbsa/cu129

See the Ubuntu 24.04 section of the docs for details and a list of available containers 🤗 Thanks to all our contributors from Discord and AI community for their support 🤗

What is CUDA ARM SBSA vs TEGRA?

• Read more about CUDA ARM SBSA and Tegra on NVIDIA Developer
• Read a summary on Jetson AI Lab

Code Style

The project uses automated code formatting tools to maintain consistent code style. See Code Style Guide for details on: - Setting up formatting tools - Adding your package to formatting checks - Troubleshooting common issues

| | | |--------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | ML | pytorch tensorflow jax onnxruntime deepstream holoscan CTranslate2 JupyterLab | | LLM | SGLang vLLM MLC AWQ transformers text-generation-webui ollama llama.cpp llama-factory exllama AutoGPTQ FlashAttention DeepSpeed bitsandbytes xformers | | VLM | llava llama-vision VILA LITA NanoLLM ShapeLLM Prismatic xtuner gemma_vlm | | VIT | NanoOWL NanoSAM Segment Anything (SAM) Track Anything (TAM) clip_trt | | RAG | llama-index langchain jetson-copilot NanoDB FAISS RAFT | | L4T | l4t-pytorch l4t-tensorflow l4t-ml l4t-diffusion l4t-text-generation | | CUDA | cupy cuda-python pycuda cv-cuda opencv:cuda numba | | Robotics | ROS LeRobot OpenVLA 3D Diffusion Policy Crossformer MimicGen OpenDroneMap ZED openpi | | Simulation | Isaac Sim Genesis Habitat Sim MimicGen MuJoCo PhysX Protomotions RoboGen RoboMimic RoboSuite Sapien | | Graphics | 3D Diffusion Policy AI Toolkit ComfyUI Cosmos Diffusers Diffusion Policy FramePack Small Stable Diffusion Stable Diffusion Stable Diffusion WebUI SD.Next nerfstudio meshlab gsplat | | Mamba | mamba mambavision cobra dimba videomambasuite | | KANs | pykan kat | | xLTSM | xltsm mlstm_kernels | | Speech | whisper whisper_trt piper riva audiocraft voicecraft xtts | | Home/IoT | homeassistant-core wyoming-whisper wyoming-openwakeword wyoming-piper | | 3DPrintObjects | PartPacker Sparc3D |

See the packages directory for the full list, including pre-built container images for JetPack/L4T.

Using the included tools, you can easily combine packages together for building your own containers. Want to run ROS2 with PyTorch and Transformers? No problem - just do the system setup, and build it on your Jetson:

bash $ jetson-containers build --name=my_container pytorch transformers ros:humble-desktop

There are shortcuts for running containers too - this will pull or build a l4t-pytorch image that's compatible:

bash $ jetson-containers run $(autotag l4t-pytorch)

^{jetson-containers run launches docker run with some added defaults (like --runtime nvidia, mounted /data cache and devices)
autotag finds a container image that's compatible with your version of JetPack/L4T - either locally, pulled from a registry, or by building it.}

If you look at any package's readme (like l4t-pytorch), it will have detailed instructions for running it.

Changing CUDA Versions

You can rebuild the container stack for different versions of CUDA by setting the CUDA_VERSION variable:

bash CUDA_VERSION=12.6 jetson-containers build transformers

It will then go off and either pull or build all the dependencies needed, including PyTorch and other packages that would be time-consuming to compile. There is a Pip server that caches the wheels to accelerate builds. You can also request specific versions of cuDNN, TensorRT, Python, and PyTorch with similar environment variables like here.

Documentation

• Package List
• Package Definitions
• System Setup
• Building Containers
• Running Containers

Check out the tutorials at the Jetson Generative AI Lab!

Getting Started

Refer to the System Setup page for tips about setting up your Docker daemon and memory/storage tuning.

```bash

install the container tools

git clone https://github.com/dusty-nv/jetson-containers bash jetson-containers/install.sh

automatically pull & run any container

jetson-containers run $(autotag l4t-pytorch) ```

Or you can manually run a container image of your choice without using the helper scripts above:

bash sudo docker run --runtime nvidia -it --rm --network=host dustynv/l4t-pytorch:r36.2.0

Looking for the old jetson-containers? See the legacy branch.

Gallery

Multimodal Voice Chat with LLaVA-1.5 13B on NVIDIA Jetson AGX Orin (container: NanoLLM)

Interactive Voice Chat with Llama-2-70B on NVIDIA Jetson AGX Orin (container: NanoLLM)

Realtime Multimodal VectorDB on NVIDIA Jetson (container: nanodb)

NanoOWL - Open Vocabulary Object Detection ViT (container: nanoowl)

Live Llava on Jetson AGX Orin (container: NanoLLM)

Live Llava 2.0 - VILA + Multimodal NanoDB on Jetson Orin (container: NanoLLM)

Small Language Models (SLM) on Jetson Orin Nano (container: NanoLLM)

Realtime Video Vision/Language Model with VILA1.5-3b (container: NanoLLM)

Citation

Please see CITATION.cff for citation information.