Tutorial for DNN Image Segmentation with NITROS Bridge

Overview

This tutorial walks you through how to run the isaac_ros_unet on ROS 2 Humble, while playing rosbag and getting the results from ROS Noetic through isaac_ros_nitros_bridge. The tutorial is based on the ROS1 Bridge documentation.

Note

ROS actions are not supported by the ROS1 Bridge.

Steps

1. Complete the Isaac ROS Nitros Bridge quickstart.

2. Complete the Isaac ROS Unet quickstart.

3. Add the --pid=host Docker argument (if not already present), then launch the Docker container:

   cd ${ISAAC_ROS_WS}/src/isaac_ros_common/scripts && \
   grep -qxF -- '--pid=host' .isaac_ros_dev-dockerargs 2>/dev/null || echo "--pid=host" >> .isaac_ros_dev-dockerargs && \
   cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
   ./scripts/run_dev.py -d ${HOME}/workspaces
   

4. Launch the image converter launch file of isaac_ros_nitros_bridge:

   cd /workspaces/isaac_ros-dev/ && \
      source install/setup.bash && \
      ros2 launch isaac_ros_nitros_bridge_ros2 isaac_ros_nitros_bridge_image_converter.launch.py sub_image_name:=unet/colored_segmentation_mask pub_image_name:=image
   

5. Attach a second terminal to the Docker container:

   cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
   ./scripts/run_dev.py -d ${HOME}/workspaces
   

6. Run the following launch files to spin up isaac_ros_unet, (you may need to re-download the unet models and convert them since now you are in a new container):

   cd /workspaces/isaac_ros-dev/ && \
      source install/setup.bash && \
      ros2 launch isaac_ros_unet isaac_ros_unet_tensor_rt.launch.py engine_file_path:=${ISAAC_ROS_WS}/isaac_ros_assets/models/peoplesemsegnet/deployable_quantized_vanilla_unet_onnx_v2.0/1/model.plan input_binding_names:=['input_1:0'] output_binding_names:=['argmax_1'] network_output_type:='argmax' input_image_width:=1200 input_image_height:=632
   

7. Open another terminal, launch the Noetic Docker container:

   docker run -it --cap-add=SYS_PTRACE --privileged --network host --pid host --ipc host --runtime nvidia --name nitros_bridge -e FASTRTPS_DEFAULT_PROFILES_FILE=/usr/local/share/middleware_profiles/rtps_udp_profile.xml --rm nitros_bridge:latest nitros_bridge_image_converter.yaml nitros_bridge_image_converter.launch /image /ros1_output_image
   

8. Attach a second terminal to the Noetic Docker container:

   docker exec -it nitros_bridge /bin/bash
   

9. Play the rosbag under ROS Noetic:

   cd /workspaces/isaac_ros_1-dev/ && \
     source install_isolated/setup.bash && \
     rosbag play -l "src/isaac_ros_nitros_bridge/resources/unet_sample_data_ros1.bag"
   

10. Attach a third terminal to the Noetic Docker container:

    docker exec -it nitros_bridge /bin/bash
    

11. Use rostopic to echo in the container:

    cd /workspaces/isaac_ros_1-dev && \
     source install_isolated/setup.bash && \
     rostopic echo /ros1_output_image
    

12. Example output use rostopic hz: