Gear Insertion Policy#

Overview#

This tutorial demonstrates a learning-based gear insertion policy using Isaac for Manipulation in Isaac Sim. The robot uses a trained policy to precisely insert a gear into a peg stand, demonstrating advanced manipulation with contact-rich tasks.

Isaac Sim scene used for gear insertion policy with the gear and peg stand objects

Prerequisites#

Tutorial#

Launch Isaac Sim#

  1. Open Isaac Sim and load the scene:

    https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.1/Isaac/Samples/ROS2/Scenario/isaac_manipulator_scene.usd

    Tip

    We recommend setting a ROS_DOMAIN_ID via export ROS_DOMAIN_ID=<ID_NUMBER> for every new terminal where you run ROS commands, to avoid interference with other computers in the same network (ROS Guide).

  2. Hit Play in Isaac Sim to start the simulation.

Configure the Gear Assembly Workflow#

  1. Open the sim_launch_params.yaml file and set the grasps_file_path to the path to the grasps file you want to use.

    grasps_file_path: $(ros2 pkg prefix --share isaac_manipulator_robot_description)/config/robotiq_2f_140_grasps_large_gear.yaml
segmentation_type: SEGMENT_ANYTHING
object_detection_type: SEGMENT_ANYTHING
enable_nvblox: 'false'
workflow_type: GEAR_ASSEMBLY

Run Gear Insertion Policy#

  1. Once you hit Play on the scene in Isaac Sim, go to the Window tab and select Script Editor.

  2. Once the robot is in the gear insertion position, you can run the gear insertion workflow as described below.

  3. Open a new terminal and activate the Isaac ROS environment:

    isaac-ros activate

  4. Run the gripper driver:

    ros2 run isaac_manipulator_isaac_sim_utils isaac_sim_gripper_driver.py

  5. In a new terminal, run the following command to open the gripper:

    ros2 action send_goal \
/robotiq_gripper_controller/gripper_cmd \
control_msgs/action/GripperCommand "command: {position: 0.0, max_effort: 100.0}"

  6. Paste contents of the script from isaac_manipulator_bringup/test/isaac_sim_script_editor_scripts/place_robot_in_gear_insertion_position.py in the Isaac Sim Script Editor.

  7. Run it and wait until the robot is in the gear insertion position.

  8. Run the script again to make sure gear is in correct position.

  9. Close the gripper via this call:

    ros2 action send_goal \
/robotiq_gripper_controller/gripper_cmd \
control_msgs/action/GripperCommand "command: {position: 0.50, max_effort: 100.0}"

  10. Launch the gear insertion workflow:

    export ENABLE_MANIPULATOR_TESTING=manual_isaac_sim
launch_test $(ros2 pkg prefix --share isaac_manipulator_bringup)/test/gear_insertion_pol_test_sim.py

  11. Wait for user input prompt to start the insertion process. Once prompted, run this command:

    ros2 topic pub /input_points_debug geometry_msgs/msg/Point "{x: 0.0, y: 0.0, z: 0.0}"

  12. The robot should insert the gear into the peg stand.

Next Steps#