OpenTera - WebRTC

WebRTC is a standard for real-time audio/video/data communication and is mostly used in web browsers. For the OpenTera micro-service architecture, we wanted to use Google's native WebRTC implementation for robots so we would have more control on the streams and develop a framework that is optimized for robot teleoperation. We also wanted to use hardware acceleration if possible on embedded platforms.

This project provides:

• A signaling server written in Python that can be used as a stand-alone server or embedded in our OpenTera Teleoperation Micro-Service
• A native libwebrtc wrapper that is used as a C++ base library for clients using our signaling server.
• GStreamer video codec factories for libwebrtc to enable both software and hardware encoders/decoders if available.
• Possiblity to force usage of a particular codec with VideoStreamConfiguration
• Python 3 bindings that are used as a Python module for clients using our signaling server.
• A JavaScript library That is used for web client developement and teleoperation interface using our signaling server.

License

By default, libwebrtc is built with non-free codecs. See webrtc-native-build to build without them.

• Apache License, Version 2.0
• For GStreamer components, please read this.

API Documentation

API Documentation is available here.

How to Contribute

Please read the Code of Conduct and CONTRIBUTING files.

Dependencies

• Most dependencies are part of the project with git submodules in the opentera-webrtc-native-client/3rdParty directory.

Build Instructions

• Please see Build Instructions.

Examples

C++

• data-channel-client
• cpp-video-stream-client
• cpp-camera-stream-client

Python

• data-channel-client
• stream-client

Javascript

• data-channel-client
• stream-client
• stream-data-channel-client

Hardware Acceleration with GStreamer

Hardware acceleration depends on the platform you are using to build opentera-webrtc. The following table summarizes supported hardware and GStreamer plugins:

| Platform | VP8 | VP9 | H.264 | |-----------------------------------|-----------------------------|---------------|------------------------------| | Jetson TX2/Nano | nvv4l2vp8enc, nvv4l2decoder | nvv4l2decoder | nvv4l2h264enc, nvv4l2decoder | | Jetson Xavier NX | X | nvv4l2decoder | nvv4l2h264enc, nvv4l2decoder | | Jetson AGX Xavier | X | nvv4l2decoder | nvv4l2h264enc, nvv4l2decoder | | Jetson Orin / Orin Nano / Orin NX | X | nvv4l2decoder | nvv4l2h264enc, nvv4l2decoder | | Raspberry Pi 4 | X | X | v4l2h264enc, v4l2h264dec | | VA-API | vaapivp8enc, vaapivp8dec | vaapivp9dec | vaapih264enc, vaapih264dec | | Apple Media | X | X | vtenc_h264, vtdec |

Note that VP9 encoding is under development, see issue 117.

ROS

• The C++ library and Python bindings are used in our openterawebrtcros ROS package. RGB images published in ROS topics can be easily used for streaming in a RTCPeerConnection using our signaling server. Hardware acceleration and codec selection can be used if available.

Frontend

• The JavaScript library is used for our opentera WebRTC teleoperation frontend.

Authors

• Marc-Antoine Maheux (@mamaheux)
• Dominic Létourneau (@doumdi)
• Philippe Warren (@philippewarren)
• François Michaud (@michaudf)

Contributors

• Cédric Godin (@godced)
• Ian-Mathieu Joly (@joli-1801)

Sponsor

IntRoLab - Intelligent / Interactive / Integrated / Interdisciplinary Robot Lab