Isaac ROS DNN Stereo Depth


Isaac ROS DNN Stereo Depth provides a GPU-accelerated package for DNN-based stereo disparity. Stereo disparity is calculated from a time-synchronized image pair sourced from a stereo camera and is used to produce a depth image or a point cloud for a scene. The isaac_ros_ess package uses the ESS DNN model to perform stereo depth estimation via continuous disparity prediction. Given a pair of stereo input images, the package generates a disparity map of the left input image.

ESS is used in a graph of nodes to provide a disparity prediction from an input left and right stereo image pair. Images to ESS need to be rectified and resized to the appropriate input resolution. The aspect ratio of the image is recommended to be maintained, so the image may need to be cropped and resized to maintain the input aspect ratio. The graph for DNN encode, DNN inference, and DNN decode is included in the ESS node. Inference is performed using TensorRT, as the ESS DNN model is designed with optimizations supported by TensorRT. ESS node is agnostic to the model dimension and disparity output has the same dimension as the ESS model.


Isaac ROS NITROS Acceleration

This package is powered by NVIDIA Isaac Transport for ROS (NITROS), which leverages type adaptation and negotiation to optimize message formats and dramatically accelerate communication between participating nodes.


ESS stands for Efficient Semi-Supervised stereo disparity, developed by NVIDIA. The ESS DNN is used to predict the disparity for each pixel from stereo camera image pairs. This network has improvements over classic CV approaches that use epipolar geometry to compute disparity, as the DNN can learn to predict disparity in cases where epipolar geometry feature matching fails. The semi-supervised learning and stereo disparity matching makes the ESS DNN robust in environments unseen in the training datasets and with occluded objects. This DNN is optimized for and evaluated with color (RGB) global shutter stereo camera images, and accuracy may vary with monochrome stereo images used in analytic computer vision approaches to stereo disparity. The predicted disparity values represent the distance a point moves from one image to the other in a stereo image pair (a.k.a. the binocular image pair). The disparity is inversely proportional to the depth (i.e. disparity = focalLength x baseline / depth). Given the focal length and baseline of the camera that generates a stereo image pair, the predicted disparity map from the isaac_ros_ess package can be used to compute depth and generate a point cloud.


Compare the requirements of your use case against the package input limitations.

DNN models

An ESS model is required to run isaac_ros_ess. NGC provides pre-trained models for use in your robotics application. ESS models are available on NGC, providing robust depth estimation.

Click here for more information on how to use NGC models.

Confidence and Density

ESS DNN provides two outputs: disparity estimation and confidence estimation. The disparity output can be filtered, by thresholding the confidence output, to trade-off between confidence and density. isaac_ros_ess filters out pixels with low confidence by setting:

disparity[confidence < threshold] = -1  # -1 means invalid

The choice of threshold value is dependent on use case.

Resolution and Performance

NGC Provides ESS 3.0 and Light ESS 3.0 models for trade-off between resolution and performance. A detailed comparison of the two models can be found here.

Model Name

Disparity Resolution


Estimate disparity at 1/4 HD resolution

Light ESS

Estimate disparity at 1/16 HD resolution


Supported Platforms

This package is designed and tested to be compatible with ROS 2 Humble running on Jetson or an x86_64 system with an NVIDIA GPU.


Versions of ROS 2 earlier than Humble are not supported. This package depends on specific ROS 2 implementation features that were only introduced beginning with the Humble release.






Jetson Orin Jetson Xavier

JetPack 5.1.2

For best performance, ensure that power settings are configured appropriately.



Ubuntu 20.04+ CUDA 11.8+


To simplify development, we strongly recommend leveraging the Isaac ROS Dev Docker images by following these steps. This will streamline your development environment setup with the correct versions of dependencies on both Jetson and x86_64 platforms.


All Isaac ROS Quickstarts, tutorials, and examples have been designed with the Isaac ROS Docker images as a prerequisite.

Customize your Dev Environment

To customize your development environment, reference this guide.





Updated for ESS 3.0 with confidence thresholding in multiple resolutions


Upgraded model (1.1.0)


Source available GXF extensions


Updated OSS licensing


Update to be compatible with JetPack 5.0.2


Initial release