Tutorial for RT-DETR with Isaac Sim

Overview

1. Detect object bounding boxes using Isaac ROS RT-DETR object detection

Note

This tutorial detects the Mac and Cheese Box mentioned in the SyntheticaDETR Object Classes section. In order to detect other objects from SyntheticaDETR Object Classes, you need to create a mesh and replace the mesh_file_path and texture_path arguments to this tutorials launch file with the mesh you generated. The tutorial on how to create a mesh using an iPhone can be found here. In order to detect other objects not supported by SyntheticaDETR, you will have to modify the launch file and replace the 2D object detection pipeline with one that supports the object you are trying to detect.

Tutorial Walkthrough

1. Complete the Isaac ROS RT-DETR tutorial.

2. Open a new terminal and launch the Docker container using the run_dev.sh script:

   cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
     ./scripts/run_dev.sh
   

3. Install and launch Isaac Sim following the steps in the Isaac ROS Isaac Sim Setup Guide.

4. Run the launch file.

   ros2 launch isaac_ros_rtdetr isaac_ros_rtdetr_isaac_sim.launch.py
   

5. Press Play to start publishing data from the Isaac Sim.

   

6. Validate that you see a visualization of the 2D object detection as shown below: