Tutorial for FoundationStereo with Isaac Sim

This tutorial walks you through a graph to estimate depth with stereo images from Isaac Sim using FoundationStereo.

Tutorial Walkthrough

1. Install and launch Isaac Sim following the steps in the Isaac ROS Isaac Sim Setup Guide.

2. Press Play to start publishing data from the Isaac Sim.

   

3. Complete the isaac_ros_foundationstereo quickstart.

Note

Use the FoundationStereo model for the quickstart.

4. Open a new terminal and attach to the container:

   isaac-ros activate
   

5. Install rviz2 (required for the launch file visualization):

   sudo apt-get install -y ros-jazzy-rviz2
   

   Source the setup file.

   source /opt/ros/jazzy/setup.bash
   

6. In the terminal, start the isaac_ros_foundationstereo node using the launch files:

   High Resolution (high_res)Low Resolution (low_res)

   ros2 launch isaac_ros_foundationstereo isaac_ros_foundationstereo_isaac_sim.launch.py \
      engine_file_path:=${ISAAC_ROS_WS:?}/isaac_ros_assets/models/foundationstereo/deployable_foundation_stereo_small_v1.0/foundationstereo_576x960.engine
   

   The launch file will automatically open RViz configured to display:

   • The 3D point cloud generated from the disparity output (/points2)

   • The left and right stereo camera images

7. Optionally, you can run the visualizer script to visualize the disparity image:

   ros2 run isaac_ros_foundationstereo isaac_ros_foundationstereo_visualizer.py