Tutorial for cuMotion MoveIt Plugin with Isaac Sim

https://media.githubusercontent.com/media/NVIDIA-ISAAC-ROS/.github/main/resources/isaac_ros_docs/concepts/manipulation/cumotion_moveit/isaac_sim.jpg/

Overview

This tutorial walks through the process of planning trajectories for a simulated Franka or UR robot in Isaac Sim leveraging the cuMotion plugin for MoveIt 2 provided by Isaac ROS cuMotion. It is based on an example developed by PickNik Robotics, which in turn was based on a MoveIt example with fewer features provided with Isaac Sim.

This tutorial has been tested with Isaac Sim 4.2.0

Tutorial Walkthrough

Set Up Development Environment

  1. Set up your development environment by following the instructions in Compute Setup and Getting Started.

  2. Clone isaac_ros_common under ${ISAAC_ROS_WS}/src.

    cd ${ISAAC_ROS_WS}/src && \
   git clone -b release-3.2 https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common.git isaac_ros_common

Install isaac_ros_cumotion_moveit and Examples

  1. Install the isaac_ros_cumotion_examples package and its dependencies (including isaac_ros_cumotion and isaac_ros_cumotion_moveit):

    1. Launch the Docker container using the run_dev.sh script:

      cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
   ./scripts/run_dev.sh

    2. Install the prebuilt Debian package:

      sudo apt-get update

      sudo apt-get install -y ros-humble-isaac-ros-cumotion-examples

  2. Install and configure Isaac Sim following the steps in the Isaac ROS Isaac Sim Setup Guide, stopping after step 4.

  3. This tutorial uses the Isaac Sim standalone workflow, in which Isaac Sim is launched via a Python script. In the terminal opened using the Open in Terminal button in the Isaac Sim App Selector (and notably not in a shell within the Isaac ROS Docker container), source your ROS 2 workspace. For example:

    source /opt/ros/humble/setup.bash

  4. Set up your robot using the instructions below:

  1. Start Isaac Sim and open the following scene:

    https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/4.2/Isaac/Samples/ROS2/Scenario/isaac_manipulator_ur10e_robotiq_2f_140.usd

  2. Optional: The above scene contains a UR10e robot. If you would like to experiment with a different UR robot, first delete the UR10e from the scene. Then find the alternative robot in Isaac Sim’s robot assets folder and drag it onto the table in the scene. Finally, edit the Action Graph so that the ROS2 Publish Joint State and Articulation Controller nodes refer to the new robot, and save the USD file.

  3. Once the scene is fully loaded, click Play.

    Note

    Before launching MoveIt, ensure that the scene is loaded in Isaac Sim and that the timeline is playing.

    Running ros2 topic list in your ROS workspace should show a number of topics streaming data from Isaac Sim.

Set Up MoveIt 2 with cuMotion

  1. Open a new terminal inside the Docker container.

    cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
   ./scripts/run_dev.sh

  2. Launch MoveIt (including RViz) using the provided launch file. Remember to first run source install/setup.bash if package was built from source.

    ros2 launch isaac_ros_cumotion_examples franka_isaac_sim.launch.py

    Verify that the visualization of the robot in RViz reflects the current joint state of the simulated robot in Isaac Sim.

  3. Open another terminal inside the Docker container.

    cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
   ./scripts/run_dev.sh

  4. Run the cuMotion planner node. Remember to first run source install/setup.bash if package was built from source.

    ros2 run isaac_ros_cumotion cumotion_planner_node --ros-args \
   -p robot:=$(ros2 pkg prefix --share isaac_ros_cumotion_robot_description)/xrdf/franka.xrdf \
   -p urdf_path:=/opt/ros/humble/share/moveit_resources_panda_description/urdf/panda.urdf

Using the cuMotion Planner in MoveIt

  1. In RViz, enable cuMotion by ensuring that isaac_ros_cumotion and cuMotion are selected within the “Planning Library” pane within the “Context” tab in the bottom left corner of the RViz window.

https://media.githubusercontent.com/media/NVIDIA-ISAAC-ROS/.github/main/resources/isaac_ros_docs/repositories_and_packages/isaac_ros_cumotion/isaac_ros_cumotion_moveit/moveit_context.png/

  1. Select a target pose for the robot end effector, and click the Plan button in the Planning tab. If planning is successful, an animation of the trajectory will be visualized in RViz.

  2. Click the Execute button in RViz. The simulated Franka robot in Isaac Sim will follow the motion of the robot in RViz.