Tutorial for People Segmentation with Segformer and Isaac Sim

Overview

This tutorial walks you through a graph for Image Segmentation of people using images from Isaac Sim.

Tutorial Walkthrough

1. Complete the quickstart up until model preparation step.

2. Launch the Docker container using the run_dev.sh script:

   cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
     ./scripts/run_dev.sh
   

3. Install and launch Isaac Sim following the steps in the Isaac ROS Isaac Sim Setup Guide.

4. Press Play to start publishing data from the Isaac Sim.

   

5. Run the following launch files to start the inferencing:

   ros2 launch isaac_ros_segformer isaac_ros_people_sem_segformer_isaac_sim.launch.py model_name:=peoplesemsegformer model_repository_paths:=[${ISAAC_ROS_WS}/isaac_ros_assets/models]
   

6. In another terminal, visualize and validate the output of the package by launching rqt_image_view:

   cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
       ./scripts/run_dev.sh
   

   Then launch rqt_image_view:

   ros2 run rqt_image_view rqt_image_view
   

   To view a colorized segmentation mask, inside the rqt_image_view GUI, change the topic to /segformer/colored_segmentation_mask.

   Note

   The raw segmentation is also published to /segformer/raw_segmentation_mask. However, the raw pixels correspond to the class labels and so the output is unsuitable for human visual inspection.