Isaac ROS Image Pipeline
Isaac ROS Image Pipeline is a metapackage of functionality for image processing. Camera output often needs pre-processing to meet the input requirements of multiple different perception functions. This can include cropping, resizing, mirroring, correcting for lens distortion, and color space conversion. For stereo cameras, additional processing is required to produce disparity between left + right images and a point cloud for depth perception.
This package is accelerated using the GPU and specialized hardware
engines for image computation, replacing the CPU-based
Considerable effort has been made to ensure that replacing
isaac_ros_image_pipeline on a Jetson or GPU
is as painless a transition as possible.
Some image pre-processing functions use specialized hardware engines, which offload the GPU to make more compute available for other tasks.
Rectify corrects for lens distortion from the received camera sensor
message. The rectified image is resized to the input resolution for
disparity, using a crop before resizing to maintain image aspect ratio.
The image is color space converted to YUV from RGB using the luma
channel (the Y in YUV) to compute disparity using
common graph of nodes can be performed without the CPU processing a
single pixel using
isaac_ros_image_pipeline; in comparison, using
image_pipeline, the CPU would process each pixel ~3 times.
The Isaac ROS Image Pipeline metapackage offloads the CPU from common image processing tasks so it can perform robotics functions best suited for the CPU.
Isaac ROS NITROS Acceleration
This package is powered by NVIDIA Isaac Transport for ROS (NITROS), which leverages type adaptation and negotiation to optimize message formats and dramatically accelerate communication between participating nodes.
This package is designed and tested to be compatible with ROS 2 Humble running on Jetson or an x86_64 system with an NVIDIA GPU.
Versions of ROS 2 earlier than Humble are not supported. This package depends on specific ROS 2 implementation features that were only introduced beginning with the Humble release.
For best performance, ensure that power settings are configured appropriately.
To simplify development, we strongly recommend leveraging the Isaac ROS Dev Docker images by following these steps. This will streamline your development environment setup with the correct versions of dependencies on both Jetson and x86_64 platforms.
All Isaac ROS Quickstarts, tutorials, and examples have been designed with the Isaac ROS Docker images as a prerequisite.
Customize your Dev Environment
To customize your development environment, reference this guide.
Introduced confidence thresholding for SGM
Improved stereo rectification
Source available GXF extensions
Updated OSS licensing
Image flip support and update to be compatible with JetPack 5.0.2
Migrated to NITROS based implementation
Migrated to NVIDIA-ISAAC-ROS. Fixed handling of extrinsics in Rectify and Disparity nodes.
Initial release to NVIDIA-AI-IOT