isaac_ros_stereo_image_proc

Source code on GitHub.

Quickstart

1. Set up your development environment by following the instructions here.

2. Clone isaac_ros_common and this repository under ${ISAAC_ROS_WS}/src.

   cd ${ISAAC_ROS_WS}/src
   

   git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common.git
   

git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_image_pipeline.git

3. Build the RealSense camera package for Humble, using this doc.

4. Launch the Docker container using the run_dev.sh script:

   cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
     ./scripts/run_dev.sh
   

5. Install this package’s dependencies.

   sudo apt-get install -y ros-humble-isaac-ros-stereo-image-proc
   

6. Start isaac_ros_stereo_image_proc using the prebuilt launch file (Using RealSense camera as an example):

   ros2 launch isaac_ros_stereo_image_proc isaac_ros_stereo_image_pipeline_realsense.launch.py
   

Note

For RealSense camera package issues, please refer to the section here.

Other supported cameras can be found here.

For camera calibration, please refer to this guide.

Try More Examples

To continue your exploration, check out the following suggested examples:

• Tutorial with Isaac Sim
• Tutorial with RealSense
• Tutorial with Zed

API

Overview

The isaac_ros_stereo_image_proc package offers functionality for handling image pairs from a binocular/stereo camera setup, calculating the disparity between the two images, and producing a point cloud with depth information. It largely replaces the stereo_image_proc package.

Available Components

Component	Topics Subscribed	Topics Published	Parameters
DisparityNode	left/image_rect, left/camera_info: The left camera stream right/image_rect, right/camera_info: The right camera stream	disparity: The disparity between the two cameras	max_disparity: The maximum value for disparity per pixel, which is 64 by default. With ORIN backend, this value must be 128 or 256. backends: The VPI backend to use, which is CUDA by default (options: “CUDA”, “XAVIER”, “ORIN”) confidence_threshold: The confidence threshold for VPI SGM algorithm window_size: The window size for SGM disparity calculation num_passes: The number of passes SGM takes to compute result p1: Penalty on disparity changes of +/- 1 between neighbor pixels p2: Penalty on disparity changes of more than 1 between neighbor pixels p2_alpha: Alpha for P2 quality: Quality of disparity output. It’s only applicable when using XAVIER backend. The higher the value, better the quality and possibly slower performance. Refer tothis VPI docfor more details on the parameters
PointCloudNode	left/image_rect_color: The coloring for the point cloud left/camera_info: The left camera info right/camera_info: The right camera info disparity The disparity between the two cameras	points2: The output point cloud	use_color: Whether or not the output point cloud should have color. The default value is false. unit_scaling: The amount to scale the xyz points by
DisparityToDepthNode	disparity The disparity image	depth The resultant depth image	N/A