Isaac ROS Compression

Overview

Isaac ROS Compression provides H.264 image encoder and decoder that leverages the specialized hardware in NVIDIA GPUs and the Jetson platform. The isaac_ros_h264_encoder package can compress an image into H.264 data using the NVENC. The isaac_ros_h264_decoder package can decode the H.264 data into original images using the NVDEC.

Image compression reduces the data footprint of images when written to storage or transmitted between computers. A 1080p camera at 30fps produces 177MB/s of data; image compression reduces this by approximately 10 times to 17MB/s of data, reducing the throughput needed to send this to another computer or write out to storage; a one minute 1080p camera recording is reduced from ~10GB to ~1GB. This compression is provided by dedicated hardware acceleration (NvEnc) separate from other hardware engines such as the GPU.

A common use case for image compression during the development of robots is to capture camera images to storage. This captured data is processed offline from the robot to produce training datasets for AI models, test datasets for perception functions, and test data for open-loop re-simulation of software in development with real data. The compression parameters are tuned to minimize visual quality reduction from lossy compression for AI model and perception function development. Compression reduces the amount of data written to storage, the time required to offload the recording, and footprint of the data at rest in a data lake.

Compression can be used with event data recorders to capture camera images to storage when an event of interest occurs, often due to failures on the robot. This provides visual information to assist in the debugging of the event or to improve perception and robot functions.

H.264 is an efficient and popular compression algorithm with broad support across many platforms. The output of the isaac_ros_h264_encoder package can then be decoded with hardware acceleration using the isaac_ros_h264_decoder on Jetson and x86_64 systems, or by third-party H.264 decoder packages on non-NVIDIA platforms.

This package is powered by NVIDIA Isaac Transport for ROS (NITROS), which leverages type adaptation and negotiation to optimize message formats and dramatically accelerate communication between participating nodes.

Note

ROS 2 relies on image_transport_plugins for CPU based compression. We recommend using isaac_ros_h264_encoder as part of the graph of nodes when capturing to a rosbag for performance benefits of NITROS; ROS 2 type adaptation used by NITROS is not supported by image_transport_plugins, resulting in more CPU load, and less encode performance.

Quickstarts

• Isaac ROS H.264 Encoder

• Isaac ROS H.264 Decoder

H.264 compared to JPEG image_transport_plugins

ROS image_transport_plugins use the JPEG standard for image compression on the CPU, where each frame is compressed spatially within the 2D image. isaac_ros_h264_encoder uses the H.264 standard for video compression, which allows it to compress more efficiently than JPEG by using more advanced spatial and temporal compression. isaac_ros_h264_encoder uses the GPU for higher performance, while offloading compute work from the CPU.

H.264 video compression of individual images, or I-frame (inter-frame) are smaller in size than equivalent quality JPEG images; H.264 uses improved compression functions for smooth areas and less artifacts in high frequencies areas of the image. H.264 adds temporal compression on a sequence of images by using a P-frame (predicted frame) from a previous I or P frame. P-frames benefit on image similarity from frame to frame as occurs from a camera stream. P-frames are ~25% the size of an I-Frame. A sequence of images is compressed to one I-Frame followed by one or more P-Frames. P-Frames require less computation improving frame rate of compression.

As a result, isaac_ros_h264_encoder can perform compression to smaller sizes with the same quality as JPEG image_transport_plugins, at higher frame rates (throughput), while simultaneously offloading the CPU from this compute for compression.

Note

Check your requirements against the package’s input limitations.

Packages

• isaac_ros_h264_decoder
  • Quickstart
  • Try More Examples
  • Troubleshooting
  • API
• isaac_ros_h264_encoder
  • Quickstart
  • Try More Examples
  • API

Supported Platforms

Node	Jetson aarch64	GPU x86_64
isaac_ros_h264_encoder	✓	✓
isaac_ros_h264_decoder	✓	✓

This package is designed and tested to be compatible with ROS 2 Humble running on Jetson or an x86_64 system with an NVIDIA GPU.

Note

Versions of ROS 2 earlier than Humble are not supported. This package depends on specific ROS 2 implementation features that were only introduced beginning with the Humble release.

Platform	Hardware	Software	Notes
Jetson	Jetson Orin Jetson Xavier	JetPack 5.1.2	For best performance, ensure that power settings are configured appropriately.
x86_64	NVIDIA GPU	Ubuntu 20.04+ CUDA 11.8+

Docker

To simplify development, we strongly recommend leveraging the Isaac ROS Dev Docker images by following these steps. This will streamline your development environment setup with the correct versions of dependencies on both Jetson and x86_64 platforms.

Note

All Isaac ROS Quickstarts, tutorials, and examples have been designed with the Isaac ROS Docker images as a prerequisite.

Customize your Dev Environment

To customize your development environment, reference this guide.

Updates

Date	Changes
2023-10-18	Added support on x86_64 for isaac_ros_h264_encoder
2023-05-25	Performance improvements
2023-04-05	P-frame encoder and Tegra decoder support
2022-10-19	Initial release