(Optional) Install dependencies for any sensors you want to use by following the sensor-specific guides.
Note
We strongly recommend installing all sensor dependencies before starting any quickstarts.
Some sensor dependencies require restarting the Isaac ROS Dev container during installation, which will interrupt the quickstart process.
Make sure to repeat this step in every terminal created inside the Docker container.
Since this package was built from source, the enclosing workspace must be sourced for
ROS to be able to find the package’s contents.
sourceinstall/setup.bash
Note
Limitations on x86_64: ESS plugins only run with GPU with sm80 and above. This limits the GPU devices on x86_64 to devices with compute_80 and above.
For CUDA compute capability details, please refer to cuda-gpus.
This tutorial requires a compatible RealSense camera from
the list of available
cameras
Ensure that you have already set up your RealSense camera using the RealSense setup tutorial. If you have not, please set up the sensor and then restart this quickstart from the beginning.
Continuing inside the Docker container, install the following dependencies:
This tutorial requires an Argus-compatible stereo camera from
the list of available
cameras.
If you have an Argus-compatible camera,
you can also use the launch file provided in this tutorial to start a
fully NITROS-accelerated stereo depth graph.
Ensure that you have already set up your Hawk camera using the Hawk setup tutorial. If you have not, please set up the sensor and then restart this quickstart from the beginning.
Continuing inside the Docker container, install the following dependencies:
Use the colconbuild--packages-up-toisaac_ros_ess command to build
isaac_ros_ess; do not use the --symlink-install option. Run
sourceinstall/setup.bash after the build.
The isaac_ros_ess package offers functionality to generate a stereo
disparity map from stereo images using a trained ESS model. Given a pair
of stereo input images, the package generates a continuous disparity
image for the left input image.
Threshold value ranges between 0.0 and 1.0 for filtering disparity with confidence. Pixels with confidence less than threshold will be marked as invalid in the disparity output. Value 0.0 means a fully dense disparity output.
image_type
string
"RGB_U8".
The input image encoding type. Supports "RGB_U8" and "BGR_U8". Note that if this parameter is not specified and there is an upstream Isaac ROS NITROS node, the type will be decided automatically.
The input left and right images must have the same dimension and
resolution, and the resolution must be no larger than
1920x1200.
Each input pair (left/image_rect, right/image_rect,
left/camera_info and right/camera_info) should have the
same timestamp; otherwise, the synchronizing module inside the
ESS Disparity Node will drop the input with smaller timestamps.
The isaac_ros_ess package outputs a disparity image with dimension same as the ESS model output dimension.
ESS Model
Output Dimension
ess.onnx
960 x 576
light_ess.onnx
480 x 288
The input images are rescaled to the ESS model input dimension
before inferencing. There are two outputs from the ESS model
with the same dimension: disparity output and confidence output.
The disparity is filtered with confidence using a pre-configured threshold.
Pixels with confidence less than the threshold is replaced with -1.0 as
invalid before the inference result is published. For fully dense disparity output
without confidence thresholding, set the threshold to 0.0.
The left and right CameraInfo are used to composite a
stereo_msgs/DisparityImage. If you only care about the disparity
image, and don’t need the baseline and focal length information, you
can pass dummy camera messages.