Tutorial for FoundationPose with Isaac Sim
==========================================

Overview
--------

1. Estimate depth using :ir_repo:`Isaac ROS ESS depth estimation <isaac_ros_dnn_stereo_depth> <isaac_ros_ess>`
   
2. Detect object bounding boxes using :ir_repo:`Isaac ROS RT-DETR object detection <isaac_ros_object_detection> <isaac_ros_rtdetr>`
   
3. Detect a 3D object using :ir_repo:`Isaac ROS FoundationPose pose estimation <isaac_ros_pose_estimation> <isaac_ros_foundationpose>`

.. note::

    This tutorial detects the 3D pose of the Mac and Cheese Box mentioned in the :ir_ngc:`SyntheticaDETR Object Classes section <teams/isaac/models/synthetica_detr>`. In order to detect other objects from ``SyntheticaDETR Object Classes``, you need to create a mesh and replace the ``mesh_file_path`` and ``texture_path`` arguments to this tutorials launch file with the mesh you generated. The tutorial on how to create a mesh using an iPhone can be found :doc:`here </concepts/pose_estimation/foundationpose/tutorial_create_your_own_mesh>`. In order to detect other objects not supported by ``SyntheticaDETR``, you will have to modify the launch file and replace the 2D object detection pipeline with one that supports the object you are trying to detect.

Tutorial Walkthrough
--------------------

1.  Complete the :ref:`Isaac ROS ESS tutorial <repositories_and_packages/isaac_ros_dnn_stereo_depth/isaac_ros_ess/index:quickstart>`.

.. note::

    When going through the :ref:`Isaac ROS ESS tutorial <repositories_and_packages/isaac_ros_dnn_stereo_depth/isaac_ros_ess/index:quickstart>`, ensure you download the ``light_ess`` model.

2.  Complete the :ref:`Isaac ROS RT-DETR tutorial <repositories_and_packages/isaac_ros_object_detection/isaac_ros_rtdetr/index:quickstart>`.

3.  Complete the
    :ref:`Isaac ROS FoundationPose Quickstart Guide <repositories_and_packages/isaac_ros_pose_estimation/isaac_ros_foundationpose/index:quickstart>`.

4.  Open a new terminal and launch the Docker container using the
    ``run_dev.sh`` script:

    .. code:: bash

       cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
         ./scripts/run_dev.sh

5.  Install and launch Isaac Sim following the steps in the :doc:`Isaac ROS Isaac Sim Setup Guide </getting_started/isaac_sim/index>`.

6.  Run the launch file.

    .. code:: bash

       ros2 launch isaac_ros_foundationpose isaac_ros_foundationpose_isaac_sim.launch.py
   
7.  Press **Play** to start publishing data from the Isaac Sim.

    .. figure:: :ir_lfs:`<resources/isaac_ros_docs/getting_started/isaac_sim_sample_scene.png>`
        :align: center
        :width: 600px

8.  Validate that you see a visualization of the 3D pose estimate as shown below:

   .. figure:: :ir_lfs:`<resources/isaac_ros_docs/concepts/pose_estimation/foundationpose/foudnationpose_isaac_sim.png>`
      :alt: Isaac ROS FoundationPose Hawk Tracking
      :align: center