Isaac ROS cuMotion

Overview

Isaac ROS cuMotion provides CUDA-accelerated manipulation capabilities for robots in ROS 2. It provides two main capabilities:

1. Motion generation for robot arms via integration of cuMotion into MoveIt 2

2. Segmentation of robots from depth streams using cuMotion’s kinematics and geometry processing functions to accurately identify and filter out parts of the robot. This allows reconstruction of obstacles in the environment without spurious contributions from the robot itself.

The key advantages of using Isaac ROS cuMotion are:

• Improved cycle times: cuMotion produces smooth, optimal-time trajectories in the presence of obstacles, generally reducing motion times compared to previous state-of-the-art planners. In cluttered environments and other challenging scenarios, cuMotion can often produce a valid trajectory when other planners might fail altogether.

• Improved planning times: cuMotion takes advantage of CUDA acceleration to produce collision-free, optimal-time trajectories in a fraction of a second.

• Avoidance of obstacles captured by depth camera(s): cuMotion optionally leverages nvblox to perform 3D reconstruction of an environment from one or more depth image streams. The environment is represented as a signed distance field (SDF) for efficient obstacle-aware planning. Support is provided for segmenting and filtering the robot itself from the depth stream(s).

• Flexibility: A modular design simplifies integration with existing ROS 2 workflows, especially those already using MoveIt 2.

Warning

Before using or developing with cuMotion or other Isaac Manipulator software, please read and familiarize yourself with the associated safety information that is provided by your robot manufacturer.

In addition, we provide the following best practices:

1. Familiarize yourself with the location of the emergency stop buttons, and be prepared to apply if necessary.

2. Before operation, ensure the working area is free of any persons or other potential hazards.

3. Before operation, alert all persons near the working area that robot operation is about to begin.

4. Before and during operation, be aware of any persons entering the vicinity around the working area, and be prepared to give necessary warnings, instructions, or take other necessary actions.

5. Take extra caution when testing or deploying new features or code.

Examples are provided for three modes of operation:

• Standalone MoveIt: MoveIt’s RViz interface allows trajectories to be planned and visualized even without a physical robot or external simulator.

• Simulated robot (Isaac Sim): Trajectories may be planned and executed on a simulated robot in Isaac Sim, allowing convenient development and rapid iteration without use of physical hardware. Simulated sensors enable testing of perception, e.g., for the purpose of collision avoidance.

• Physical robot: For on-robot testing and final deployment, trajectories are planned and executed on a physical robot.

The Isaac ROS cuMotion repository currently contains the following packages:

isaac_ros_cumotion:

This package contains the cuMotion planner node and the robot segmentation node.

isaac_ros_cumotion_examples:

This package contains various examples demonstrating use of cuMotion with MoveIt.

isaac_ros_cumotion_moveit:

This package provides a plugin for MoveIt 2 that exposes cuMotion as an external planner, leveraging isaac_ros_cumotion.

Isaac ROS cuMotion is also featured as part of Isaac Manipulator.

Quickstarts

• Isaac ROS cuMotion MoveIt Plugin

• Isaac ROS cuMotion Robot Segmentation

• Isaac ROS cuMotion Object Attachment

Packages

• isaac_ros_cumotion
  • Motion Generation
  • Robot Segmentation
    • Quickstart
  • Troubleshooting
    • Isaac ROS Troubleshooting
  • API
    • CumotionActionServer
    • CumotionRobotSegmenter

• isaac_ros_cumotion_moveit
  • Quickstart
  • Try More Examples
  • Troubleshooting
• isaac_ros_esdf_visualizer
  • Overview
  • API
• isaac_ros_moveit_goal_setter
  • Overview
  • API
• isaac_ros_moveit_goal_setter_interfaces
• isaac_ros_cumotion_object_attachment
  • Object Attachment
  • Troubleshooting
  • API

Supported Platforms

This package is designed and tested to be compatible with ROS 2 Humble running on Jetson or an x86_64 system with an NVIDIA GPU.

Note

Versions of ROS 2 other than Humble are not supported. This package depends on specific ROS 2 implementation features that were introduced beginning with the Humble release. ROS 2 versions after Humble have not yet been tested.

Platform	Hardware	Software	Notes
Jetson	Jetson Orin	JetPack 6.1	For best performance, ensure that power settings are configured appropriately. Jetson Orin Nano 4GB may not have enough memory to run many of the Isaac ROS packages and is not recommended.
x86_64	Ampere or higher NVIDIA GPU Architecture with 8 GB RAM or higher	Ubuntu 22.04+	CUDA 12.6+

Docker

To simplify development, we strongly recommend leveraging the Isaac ROS Dev Docker images by following these steps. This streamlines your development environment setup with the correct versions of dependencies on both Jetson and x86_64 platforms.

Note

All Isaac ROS Quickstarts, tutorials, and examples have been designed with the Isaac ROS Docker images as a prerequisite.

Customize your Dev Environment

To customize your development environment, reference this guide.

Updates

Date	Changes
2024-12-10	Added object following and pick-and-place workflows
2024-09-26	Updated for Isaac ROS 3.1
2024-05-30	Initial release