Tutorial for DNN Image Segmentation with Isaac Sim

Overview

This tutorial walks you through a graph for Image Segmentation of people using images from Isaac Sim.

Tutorial Walkthrough

1. Complete the quickstart until the Prepare PeopleSemSegnet Model section.

2. Launch the Docker container using the run_dev.sh script:

   cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
     ./scripts/run_dev.sh
   

3. Install and launch Isaac Sim following the steps in the Isaac ROS Isaac Sim Setup Guide.

4. Press Play to start publishing data from the Isaac Sim.

   

5. Run the following launch files to start the inferencing:

   ros2 launch isaac_ros_unet isaac_ros_unet_tensor_rt_isaac_sim.launch.py engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/peoplesemsegnet/deployable_quantized_vanilla_unet_onnx_v2.0/1/model.plan input_binding_names:=['input_1:0']
   

6. In another terminal, visualize and validate the output of the package by launching rqt_image_view:

   cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
       ./scripts/run_dev.sh
   

   Then launch rqt_image_view:

   ros2 run rqt_image_view rqt_image_view
   

   To view a colorized segmentation mask, inside the rqt_image_view GUI, change the topic to /unet/colored_segmentation_mask.

   Note

   The raw segmentation is also published to /unet/raw_segmentation_mask. However, the raw pixels correspond to the class labels and so the output is unsuitable for human visual inspection.