Tutorial for ESS with Hawk Camera in Wide FoV Mode

Overview

This tutorial demonstrates how to:

Stream stereo images using Hawk camera in wide FoV mode.
Estimate depth using Isaac ROS ESS depth estimation.

Note

This tutorial requires an Argus-compatible stereo camera from the list of available cameras.

Isaac ROS NITROS Acceleration

This package is powered by NVIDIA Isaac Transport for ROS (NITROS), which leverages type adaptation and negotiation to optimize message formats and dramatically accelerate communication between participating nodes.

graph LR; argus_node("ArgusStereoNode (Raw Image)" with wide_fov enabled) --> left_rectify_node("RectifyNode (Rectified Image)"); argus_node --> right_rectify_node("RectifyNode (Rectified Image)"); left_rectify_node --> left_crop_node; left_rectify_node --> left_resize_node("ResizeNode (Resized Image)"); left_crop_node --> ess_node("ESSDisparityNode (DNN Inference)"); right_rectify_node --> right_crop_node; right_crop_node --> ess_node; left_resize_node --> point_cloud_point("PointCloudNode (Point Cloud Output)"); ess_node --> point_cloud_point;

Wide FoV mode is turned on in argus_node, which generates proper camera info for rectification. Images are cropped to remove the black spots then fed into the ESSDisparityNode for depth estimation.

If you have an Argus-compatible camera, you can also use the launch file provided in this tutorial to start a fully NITROS-accelerated stereo depth graph.

Tutorial Walkthrough

Complete the Hawk setup tutorial.
Complete session Prepare ESS Pre-trained Model in ESS Quickstart Guide.
Open a new terminal and launch the Docker container using the run_dev.sh script:
```
cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
  ./scripts/run_dev.sh
```

Build and source the workspace:

cd /workspaces/isaac_ros-dev && \
  colcon build --symlink-install && \
  source install/setup.bash

Follow the session Run Launch File in ESS Quickstart Guide with wide FoV launch file below:

Run the launch file, which launches the example and waits for 10 seconds:

ros2 launch isaac_ros_ess isaac_ros_argus_ess_wide_fov.launch.py \
   engine_file_path:=${ISAAC_ROS_WS:?}/isaac_ros_assets/models/dnn_stereo_disparity/dnn_stereo_disparity_v4.0.0/ess.engine \
   threshold:=0.35