🧍‍♂️ Pose-Inference

A high-performance, multi-threaded C++ pipeline for real-time multi-camera keypoint detection.

Developed as part of my PhD thesis, this module enables 3D human pose estimation from bounding box proposals generated by my detection pipeline.

This module supports deployment in robotic systems for real-time tracking and perception and is part of my ROS/ROS2 real-time 3D tracker and its docker-implementation.

🧪 Test results

• Intel(R) Xeon(R) W-2145 CPU @ 3.70GHz, Nvidia 2080 super, Ubuntu 20.04, CUDA 11.8, TensorRT 8.6.1.6, OpenCV 4.10.0 with RTMPose and BATCH_SIZE of 5 -> Preprocess: ~1ms, NN inference ~4ms, Postprocess: ~1ms (1000 samples)
• AMD Ryzen 9 7900X3D CPU @ 4.40GHz, Nvidia 4070 super, Ubuntu 20.04, CUDA 12.4, TensorRT 10.9.0.34, OpenCV 4.10.0 with Yolov8 and BATCH_SIZE of 5 -> Preprocess: <1ms, NN inference ~2ms, Postprocess: ~<1ms (1000 samples)

📑 Citation

If you use this software, please use the GitHub “Cite this repository” button at the top(-right) of this page.

Environment

This repository is designed to run inside the Docker 🐳 container provided here:
OpenCV-TRT-DEV

It includes all necessary dependencies (CUDA, cuDNN, OpenCV, TensorRT, CMake).

Prerequisites

In addition to the libraries installed in the container, this project relies on:

• 📦 tensorrt-cpp-api (fork)
  (Originally by cyrusbehr)
• 🧵 cpp-utils
  (Handles multithreading, JSON config parsing, and utility tools)

Environment Variables

Set the required variables (usually done via .env or your shell):

bash OPENCV_VERSION=4.10.0 # Your installed OpenCV version N_CAMERAS=5 # Optional: sets system-wide batch size

If N_CAMERAS is not set, CMake will default to a batch size of 5.

Use the trt.sh script in ./scripts to convert your .onnx model to a fixed batch size.

Notes

• The batch size is treated as a hardware constraint, defined by the number of connected cameras.
• You can change the default batch size in CMakeLists.txt to fit your system.
• Although this repo is optimized for YOLOv8 models, you can modify the post-processing stage to support any ONNX-compatible detection model.

Installation

Run the build and installation script:

bash sudo ./build_install.sh

This will configure the build system, compile the inference pipeline, and generate the binaries.

Usage

Before using the pipeline, ensure the following:

Environment Variables

These should be defined in your .env file or shell environment:

bash OPENCV_VERSION=4.10.0 # Your installed OpenCV version N_CAMERAS=5 # Optional: sets batch size (defaults to 5)

If N_CAMERAS is not set, the system assumes a default of 5 cameras.

🧠 Model Requirements

This repo is designed for trained RTMPose models exported as .onnx. The model must be exported with a fixed batch size matching your multi-camera setup.

CAdapt the configuration files in the cfg/ folder to reflect your system and model setup.

You can change the default batch size in CMakeLists.txt if needed.

Executables

Benchmark

After configuring your setup:

bash ./build/inference_benchmark

This runs the inference pipeline, processes multi-camera input, and saves images with overlayed bounding boxes and labels to the inputs/ folder.

Video Inference Export

This executable iterates over a directory of synchronized .mp4 videos and saves the result for each video in a .json file.

This example usage assumes .mp4 videos in an arbitrary ./test directory

bash ./build/video_inference_export test

BBox Overlay

This executable iterates over a directory of synchronized .mp4 videos and exported inference results (from ./build/video_inference_export). It generates new .mp4 videos with detections and a tiled video similar to the .gif in this readme.

This example usage assumes .mp4 videos and .json files in an arbitrary ./test directory

bash ./build/bbox_overlay test

📷 Applications

This inference module is optimized for:

• 3D multi-camera human pose estimation
• Online tracking and interaction
• Real-time robotics perception pipelines