Isaac ROS Image Pipeline

Overview

Isaac ROS Image Pipeline is a metapackage of functionality for image processing. Camera output often needs pre-processing to meet the input requirements of multiple different perception functions. This can include cropping, resizing, mirroring, correcting for lens distortion, and color space conversion. For stereo cameras, additional processing is required to produce disparity between left + right images and a point cloud for depth perception.

This package is accelerated using the GPU and specialized hardware engines for image computation, replacing the CPU-based image_pipeline metapackage. Considerable effort has been made to ensure that replacing image_pipeline with isaac_ros_image_pipeline on a Jetson or GPU is as painless a transition as possible.

Note

Some image pre-processing functions use specialized hardware engines, which offload the GPU to make more compute available for other tasks.

Rectify corrects for lens distortion from the received camera sensor message. The rectified image is resized to the input resolution for disparity, using a crop before resizing to maintain image aspect ratio. The image is color space converted to YUV from RGB using the luma channel (the Y in YUV) to compute disparity using SGM. This common graph of nodes can be performed without the CPU processing a single pixel using isaac_ros_image_pipeline; in comparison, using image_pipeline, the CPU would process each pixel ~3 times.

The Isaac ROS Image Pipeline metapackage offloads the CPU from common image processing tasks so it can perform robotics functions best suited for the CPU.

Quickstarts

• Isaac ROS Image Proc

• Isaac ROS Stereo Image Proc

Isaac ROS NITROS Acceleration

This package is powered by NVIDIA Isaac Transport for ROS (NITROS), which leverages type adaptation and negotiation to optimize message formats and dramatically accelerate communication between participating nodes.

Packages

• isaac_ros_depth_image_proc
  • API
• isaac_ros_image_pipeline
  • Replacing image_pipeline with isaac_ros_image_pipeline
• isaac_ros_image_proc
  • Quickstart
• isaac_ros_stereo_image_proc
  • Quickstart
  • Try More Examples
  • API

Supported Platforms

This package is designed and tested to be compatible with ROS 2 Jazzy running on Jetson or an x86_64 system with an NVIDIA GPU.

Platform	Hardware	Software	Storage	Notes
Jetson	Jetson Thor	JetPack 7.0	128+ GB NVMe SSD	For best performance, ensure that power settings are configured appropriately.
x86_64	Ampere or higher NVIDIA GPU Architecture with 8 GB RAM or higher	Ubuntu 24.04	32+ GB disk space available	CUDA 13.0+ NVIDIA Driver 580+

Isaac ROS Environment

To simplify development, we strongly recommend leveraging the Isaac ROS CLI by following these steps. This streamlines your development environment setup with the correct versions of dependencies on both Jetson and x86_64 platforms.

Note

All Isaac ROS Quickstarts, tutorials, and examples have been designed with the Isaac ROS CLI-managed environment as a prerequisite.

Customize your Dev Environment

To customize your development environment, refer to this guide.

Updates

Date	Changes
2025-10-24	Support for ROS 2 Jazzy with VPI 4.0
2024-12-10	Added new alpha blending node
2024-09-26	Update for ZED compatibility
2024-05-30	Update to be compatible with JetPack 6.0
2023-10-18	Introduced confidence thresholding for SGM
2023-05-25	Improved stereo rectification
2023-04-05	Source available GXF extensions
2022-10-19	Updated OSS licensing
2022-08-31	Image flip support and update to be compatible with JetPack 5.0.2
2022-06-30	Migrated to NITROS based implementation
2021-10-20	Migrated to NVIDIA-ISAAC-ROS. Fixed handling of extrinsics in Rectify and Disparity nodes.
2021-08-11	Initial release to NVIDIA-AI-IOT