Gear Assembly Workflow#

Overview#

This tutorial builds on the Gear Insertion Policy tutorial and demonstrates a complex gear assembly workflow using Isaac for Manipulation in Isaac Sim. The robot picks up gears and inserts them into a peg stand in sequence, demonstrating precise manipulation and assembly operations. We use ground truth poses for the gear objects and perfect depth from the simulation.

The example uses RViz for visualization. RViz is the default visualization tool when Isaac for Manipulation is running on the same computer that is displaying the visualization.

Isaac Sim scene used for gear assembly with the gear and peg stand objects

Prerequisites#

Tutorial#

Launch Isaac Sim#

  1. Open Isaac Sim and load the scene:

    https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.1/Isaac/Samples/ROS2/Scenario/isaac_manipulator_scene.usd

    Tip

    We recommend setting a ROS_DOMAIN_ID via export ROS_DOMAIN_ID=<ID_NUMBER> for every new terminal where you run ROS commands, to avoid interference with other computers in the same network (ROS Guide).

  2. Hit Play in Isaac Sim to start the simulation.

How to use cuMotion controller and the Insertion policy controller in Isaac Sim#

  1. The cuMotion controller uses ros2_topic_based_controller which takes the last joint state set point and publishes that to the simulation.

  2. This is not ideal since when we need to run the insertion policy and control the robot, those set points are lost by the messages being published by the ros2_topic_based_controller.

  3. Hence, we fork the ros2_topic_based_controller and add the ability to shut down the controller via a topic based message.

  4. The forked repository is available here.

Configure the Gear Assembly Workflow#

  1. Open the sim_launch_params.yaml file and set the grasps_file_path to the path to the grasps file you want to use.

    grasps_file_path: $(ros2 pkg prefix --share isaac_manipulator_robot_description)/config/robotiq_2f_140_grasps_large_gear.yaml
segmentation_type: SEGMENT_ANYTHING
object_detection_type: SEGMENT_ANYTHING
enable_nvblox: 'false'
workflow_type: GEAR_ASSEMBLY

  2. Set the use_ground_truth_pose_in_sim to true if you want to use ground truth poses for the gear insertion.

  3. Set the gear_assembly_use_ground_truth_pose_in_sim to true if you want to use ground truth poses for the gear insertion.

Run Gear Assembly Workflow#

  1. Once you hit Play on the scene in Isaac Sim, go to the Window tab and select Script Editor.

  2. Copy and paste the script inside of the below mentioned path and run it.

    $(ros2 pkg prefix --share isaac_manipulator_bringup)/test/isaac_sim_script_editor_scripts/get_robot_in_home_position.py

  3. Once the robot is in the home position, you can run the gear assembly workflow as described below.

  4. Open a new terminal and activate the Isaac ROS environment:

    isaac-ros activate

  5. Launch the gear assembly workflow:

    ros2 launch isaac_manipulator_bringup workflows.launch.py \
    manipulator_workflow_config:=$(ros2 pkg prefix --share isaac_manipulator_bringup)/params/sim_launch_params.yaml

  6. Trigger the gear assembly workflow after seeing the prompt that signifies that cuMotion is ready for planning calls:

    ros2 action send_goal /gear_assembly isaac_manipulator_interfaces/action/GearAssembly {}

  7. The robot will pick up the gears and insert them into the peg stand in sequence. Note that we use ground truth poses for the gear insertion via the use_ground_truth_pose_in_sim parameter which we set to true in the manipulator configuration file.

  8. The workflow should end after inserting the medium gear.

Warning

This simulation tutorial exercises experimental work that may encounter issues such as the grasp being unstable causing the gripper to drop the gear, picking at the wrong gear, and failing insertion. Many of these issues can be traced to the wrong pose information being propagated in simulation, however, and do not manifest for the real robot workflows.

Next Steps#