isaac_ros_dnn_stereo_decoder

Source code available on GitHub.

Quickstart

Set Up Development Environment

1. Set up your development environment by following the instructions in getting started.

2. (Optional) Install dependencies for any sensors you want to use by following the sensor-specific guides.

   Note

   We strongly recommend installing all sensor dependencies before starting any quickstarts. Some sensor dependencies require restarting the development environment during installation, which will interrupt the quickstart process.

Build isaac_ros_dnn_stereo_decoder

Binary PackageBuild from Source

1. Activate the Isaac ROS environment:

   isaac-ros activate
   

2. Install the prebuilt Debian package:

   sudo apt-get update
   

   sudo apt-get install -y ros-jazzy-isaac-ros-dnn-stereo-decoder
   

Run Launch File

1. Continuing inside the Isaac ROS environment, install the following dependencies:

   sudo apt-get update
   

   sudo apt-get install -y ros-jazzy-isaac-ros-examples
   

2. Run the following launch file to start the decoder node:

   ros2 launch isaac_ros_examples isaac_ros_examples.launch.py \
      launch_fragments:=dnn_stereo_decoder \
      min_disparity:=0.0 max_disparity:=10000.0
   

Visualize Output

1. Open a new terminal and attach to the container:

   isaac-ros activate
   

2. In the terminal, visualize the disparity output using the visualizer script:

   ros2 run isaac_ros_dnn_stereo_decoder isaac_ros_dnn_stereo_visualizer.py
   

Troubleshooting

Tensor and camera info timestamp mismatch

Symptom

Warning messages about timestamp mismatch between tensor and camera info:

[dnn_stereo_decoder] Tensor and camera info pair dropped due to timestamp mismatch - consider enabling 'cache_camera_info' parameter to process all tensor messages

Solution

Enable camera info caching by setting the cache_camera_info parameter to true. This mode caches the latest camera info and processes every tensor message independently without requiring exact timestamp synchronization.

ros2 launch isaac_ros_examples isaac_ros_examples.launch.py \
   launch_fragments:=dnn_stereo_decoder cache_camera_info:=true

Isaac ROS Troubleshooting

For solutions to problems with Isaac ROS, refer to Troubleshooting.

API

Overview

The isaac_ros_dnn_stereo_decoder package provides a node that converts a disparity tensor output from any DNN stereo model into a NitrosDisparityImage. The node subscribes to a disparity tensor and camera info, applies GPU-accelerated filtering, and publishes the resulting disparity image.

Usage

ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=dnn_stereo_decoder

DNNStereoDecoderNode

ROS Parameters

ROS Parameter	Type	Default	Description
disparity_tensor_name	string	"disparity"	Name of the disparity tensor in the input TensorList message.
confidence_tensor_name	string	"" (empty)	Optional name of a confidence tensor. If set and confidence_threshold > 0, pixels with confidence below the threshold are zeroed.
confidence_threshold	double	0.0	Confidence threshold in range [0.0, 1.0] for masking low-confidence pixels. Only used if confidence_tensor_name is set.
min_disparity	double	0.0	Minimum valid disparity value. Values below this are set to 0.
max_disparity	double	10000.0	Maximum valid disparity value. Values above this are set to 0.
cache_camera_info	bool	false	If true, cache camera info and process every tensor message independently. If false, use exact time synchronization.
input_qos	string	"DEFAULT"	QoS profile for input subscribers.
output_qos	string	"DEFAULT"	QoS profile for output publisher.

ROS Topics Subscribed

ROS Topic	Interface	Description
tensor_sub	isaac_ros_tensor_list_interfaces/TensorList	Input tensor list containing the disparity tensor (and optionally confidence tensor).
right/camera_info	sensor_msgs/CameraInfo	The right camera model used to populate disparity parameters (focal length and baseline).

ROS Topics Published

ROS Topic	Interface	Description
disparity	NitrosDisparityImage	The processed NITROS disparity image with focal length and baseline parameters.

Input Tensor Requirements

1. The disparity tensor must be a float32 tensor.

2. Supported tensor shapes:

   • Rank-3: [H, W, 1] or similar

   • Rank-4: [N, C, H, W] (batch dimension ignored)

3. If using confidence masking, the confidence tensor must have the same H×W dimensions as the disparity tensor.

Output Interpretations

1. The output disparity image has the same dimensions as the input disparity tensor.

2. Disparity parameters are derived from the right camera info:

   • focal_length = P[0] (first element of projection matrix)

   • baseline = -P[3] / P[0] (from projection matrix)

   • min_disparity and max_disparity from node parameters

3. Invalid disparity values are set to 0:

   • NaN or Inf values

   • Values outside the [min_disparity, max_disparity] range

   • Values with confidence below confidence_threshold (if confidence masking is enabled)