Tutorial for ESS with Hawk Camera in Wide FoV Mode#

Overview#

This tutorial demonstrates how to:

Note

This tutorial requires an Argus-compatible stereo camera from the list of available cameras.

Isaac ROS NITROS Acceleration#

This package is powered by NVIDIA Isaac Transport for ROS (NITROS), which leverages type adaptation and negotiation to optimize message formats and dramatically accelerate communication between participating nodes.

graph LR; argus_node("ArgusStereoNode (Raw Image)" with wide_fov enabled) --> left_rectify_node("RectifyNode (Rectified Image)"); argus_node --> right_rectify_node("RectifyNode (Rectified Image)"); left_rectify_node --> left_crop_node; left_rectify_node --> left_resize_node("ResizeNode (Resized Image)"); left_crop_node --> ess_node("ESSDisparityNode (DNN Inference)"); right_rectify_node --> right_crop_node; right_crop_node --> ess_node; left_resize_node --> point_cloud_point("PointCloudNode (Point Cloud Output)"); ess_node --> point_cloud_point;

Wide FoV mode is turned on in argus_node, which generates proper camera info for rectification. Images are cropped to remove the black spots then fed into the ESSDisparityNode for depth estimation.

If you have an Argus-compatible camera, you can also use the launch file provided in this tutorial to start a fully NITROS-accelerated stereo depth graph.

Tutorial Walkthrough#

  1. Complete the Hawk setup tutorial.

  2. Complete session Prepare ESS Pre-trained Model in ESS Quickstart Guide.

  3. Open a new terminal and launch the Docker container using the run_dev.py script:

    cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
  ./scripts/run_dev.py

  4. Build and source the workspace:

    cd /workspaces/isaac_ros-dev && \
  colcon build --symlink-install && \
  source install/setup.bash

  5. Follow the session Run Launch File in ESS Quickstart Guide with wide FoV launch file below:

    Run the launch file, which launches the example and waits for 10 seconds:

    ros2 launch isaac_ros_ess isaac_ros_argus_ess_wide_fov.launch.py \
   engine_file_path:=${ISAAC_ROS_WS:?}/isaac_ros_assets/models/dnn_stereo_disparity/dnn_stereo_disparity_v4.1.0_onnx/ess.engine \
   threshold:=0.4