Tutorial for ESS with Hawk Camera in Wide FoV Mode

Overview

This tutorial demonstrates how to:

Note

This tutorial requires an Argus-compatible stereo camera from the list of available cameras.

Isaac ROS NITROS Acceleration

This package is powered by NVIDIA Isaac Transport for ROS (NITROS), which leverages type adaptation and negotiation to optimize message formats and dramatically accelerate communication between participating nodes.

graph LR; argus_node("ArgusStereoNode (Raw Image)" with wide_fov enabled) --> left_rectify_node("RectifyNode (Rectified Image)"); argus_node --> right_rectify_node("RectifyNode (Rectified Image)"); left_rectify_node --> left_crop_node; left_rectify_node --> left_resize_node("ResizeNode (Resized Image)"); left_crop_node --> ess_node("ESSDisparityNode (DNN Inference)"); right_rectify_node --> right_crop_node; right_crop_node --> ess_node; left_resize_node --> point_cloud_point("PointCloudNode (Point Cloud Output)"); ess_node --> point_cloud_point;

Wide FoV mode is turned on in argus_node, which generates proper camera info for rectification. Images are cropped to remove the black spots then fed into the ESSDisparityNode for depth estimation.

If you have an Argus-compatible camera, you can also use the launch file provided in this tutorial to start a fully NITROS-accelerated stereo depth graph.

Tutorial Walkthrough

  1. Complete the Hawk setup tutorial.

  2. Complete session Prepare ESS Pre-trained Model in ESS Quickstart Guide.

  3. Open a new terminal and launch the Docker container using the run_dev.sh script:

    cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
  ./scripts/run_dev.sh

  4. Build and source the workspace:

    cd /workspaces/isaac_ros-dev && \
  colcon build --symlink-install && \
  source install/setup.bash

  5. Follow the session Run Launch File in ESS Quickstart Guide with wide FoV launch file below:

    Run the launch file, which launches the example and waits for 10 seconds:

    ros2 launch isaac_ros_ess isaac_ros_argus_ess_wide_fov.launch.py \
   engine_file_path:=${ISAAC_ROS_WS:?}/isaac_ros_assets/models/dnn_stereo_disparity/dnn_stereo_disparity_v4.1.0_onnx/ess.engine \
   threshold:=0.4