Isaac ROS Compression
Overview
Isaac ROS Compression provides H.264 image encoder
and decoder that leverages the specialized hardware in NVIDIA GPUs and the
Jetson platform.
The isaac_ros_h264_encoder package can compress an image into H.264
data using the NVENC. The
isaac_ros_h264_decoder package can decode the H.264 data into
original images using the NVDEC.
Image compression reduces the data footprint of images when written to storage or transmitted between computers. A 1080p camera at 30fps produces 177MB/s of data; image compression reduces this by approximately 10 times to 17MB/s of data, reducing the throughput needed to send this to another computer or write out to storage; a one minute 1080p camera recording is reduced from ~10GB to ~1GB. This compression is provided by dedicated NVIDIA acceleration (NvEnc) separate from other hardware engines such as the GPU.
A common use case for image compression during the development of robots is to capture camera images to storage. This captured data is processed offline from the robot to produce training datasets for AI models, test datasets for perception functions, and test data for open-loop re-simulation of software in development with real data. The compression parameters are tuned to minimize visual quality reduction from lossy compression for AI model and perception function development. Compression reduces the amount of data written to storage, the time required to offload the recording, and footprint of the data at rest in a data lake.
Compression can be used with event data recorders to capture camera images to storage when an event of interest occurs, often due to failures on the robot. This provides visual information to assist in the debugging of the event or to improve perception and robot functions.
H.264 is an
efficient and popular compression algorithm with broad support across
many platforms. The output of the isaac_ros_h264_encoder package can then
be decoded with NVIDIA acceleration using the
isaac_ros_h264_decoder on Jetson and x86_64 systems, or by
third-party H.264 decoder packages on non-NVIDIA platforms.
This package is powered by NVIDIA Isaac Transport for ROS (NITROS), which leverages type adaptation and negotiation to optimize message formats and dramatically accelerate communication between participating nodes.
Note
ROS 2 relies on image_transport_plugins for CPU based compression.
We recommend using isaac_ros_h264_encoder as part of the graph of
nodes when capturing to a rosbag for performance benefits of NITROS;
ROS 2 type adaptation used by NITROS is not supported by image_transport_plugins,
resulting in more CPU load, and less encode performance.
Quickstarts
H.264 compared to JPEG image_transport_plugins
ROS image_transport_plugins use the JPEG standard for image compression on the CPU, where each frame is compressed spatially within the 2D image. isaac_ros_h264_encoder uses the H.264 standard for video compression, which allows it to compress more efficiently than JPEG by using more advanced spatial and temporal compression. isaac_ros_h264_encoder uses the GPU for higher performance, while offloading compute work from the CPU.
H.264 video compression of individual images, or I-frame (inter-frame) are smaller in size than equivalent quality JPEG images; H.264 uses improved compression functions for smooth areas and less artifacts in high frequencies areas of the image. H.264 adds temporal compression on a sequence of images by using a P-frame (predicted frame) from a previous I or P frame. P-frames benefit on image similarity from frame to frame as occurs from a camera stream. P-frames are ~25% the size of an I-Frame. A sequence of images is compressed to one I-Frame followed by one or more P-Frames. P-Frames require less computation improving frame rate of compression.
As a result, isaac_ros_h264_encoder can perform compression to smaller sizes with the same quality as JPEG image_transport_plugins, at higher frame rates (throughput), while simultaneously offloading the CPU from this compute for compression.
Note
Check your requirements against the package’s input limitations.
Packages
Supported Platforms
Node |
Jetson |
GPU x86_64 |
|---|---|---|
|
✓ |
✓ |
|
✓ |
✓ |
This package is designed and tested to be compatible with ROS 2 Humble running on Jetson or an x86_64 system with an NVIDIA GPU.
Note
Versions of ROS 2 other than Humble are not supported. This package depends on specific ROS 2 implementation features that were introduced beginning with the Humble release. ROS 2 versions after Humble have not yet been tested.
Platform |
Hardware |
Software |
Notes |
|---|---|---|---|
Jetson |
For best performance, ensure that power settings are configured appropriately. Jetson Orin Nano 4GB may not have enough memory to run many of the Isaac ROS packages and is not recommended. |
||
x86_64 |
|
Docker
To simplify development, we strongly recommend leveraging the Isaac ROS Dev Docker images by following these steps. This will streamline your development environment setup with the correct versions of dependencies on both Jetson and x86_64 platforms.
Note
All Isaac ROS Quickstarts, tutorials, and examples have been designed with the Isaac ROS Docker images as a prerequisite.
Customize your Dev Environment
To customize your development environment, reference this guide.
Updates
Date |
Changes |
|---|---|
2024-09-26 |
Update for ZED compatibility |
2024-05-30 |
Update to be compatible with JetPack 6.0 |
2023-10-18 |
Added support on |
2023-05-25 |
Performance improvements |
2023-04-05 |
P-frame encoder and Tegra decoder support |
2022-10-19 |
Initial release |