Tutorial for Training and Deploying an RL Policy from Isaac Lab to a Real Robot

Overview

This tutorial guides you through the process of deploying a basic RL reach policy trained in Isaac Lab on the UR10e manipulator using Isaac ROS.

Prerequisites

• Follow the setup instructions in Setup Hardware and Software for Real Robot.

Tutorial

Train and Validate Policy using Isaac Lab

1. Set up Isaac Lab using the local installation.

2. Train a policy in the Isaac-Deploy-Reach-UR10e-ROS-Inference-v0 environment with RSL-RL:

   ./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py \
      --task Isaac-Deploy-Reach-UR10e-ROS-Inference-v0 --headless
   

3. Validate the policy in simulation:

   ./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/play.py \
      --task Isaac-Deploy-Reach-UR10e-ROS-Inference-v0 --num_envs 1 --checkpoint <CHECKPOINT>
   

   Replace <CHECKPOINT> with the path to the .pth policy checkpoint file.

Deploy the Policy using Isaac ROS

1. Set up your development environment using the instructions in getting started.

2. Install isaac_manipulator_ur_dnn_policy

   Binary PackageBuild from Source

   1. Activate the Isaac ROS environment:

      isaac-ros activate
      

   2. Install the prebuilt Debian package:

      sudo apt-get update
      

      sudo apt-get install -y ros-jazzy-isaac-manipulator-ur-dnn-policy
      

3. Start the UR driver:

   ros2 launch ur_robot_driver ur_control.launch.py ur_type:=<UR_TYPE> robot_ip:=<ROBOT_IP> initial_joint_controller:=impedance_controller launch_rviz:=False kinematics_params_file:=<calibration_file_path>
   

   Replace <UR_TYPE> with the type of your UR robot (for example, ur10e) and <ROBOT_IP> with the IP address of your robot.

   Note

   Replace <calibration_file_path> with the path to the calibration file for your robot. To calibrate your UR robot, refer to the ur_calibration usage.

4. In a separate terminal, run the inference pipeline:

   ros2 launch isaac_manipulator_ur_dnn_policy inference.launch.py checkpoint:=<CHECKPOINT>
   

   Replace <CHECKPOINT> with the path to the .pth policy checkpoint file.

5. In a separate terminal, set a target end-effector pose:

   POSITION="{x: <X>, y: <Y>, z: <Z>}"; ORIENTATION="{w: <QW>, x: <QX>, y: <QY>, z: <QZ>}"
   

   Replace <X>, <Y>, and <Z> with the target end-effector position and <QW>, <QX>, <QY>, and <QZ> with the target end-effector orientation.

6. Publish the target end-effector pose:

   ros2 topic pub -r 60 /goal_pose geometry_msgs/msg/PoseStamped \
      "{ header: { stamp: now, frame_id: base }, pose: { position: $POSITION, orientation: $ORIENTATION } }"
   

7. Press play on the Teach Pendant to move the robot to the target end-effector position.

Troubleshooting

Out of distribution start state of the robot before reach

The policy for reach is trained with only a specific number of Inverse Kinematics (IK) starting solutions.

The other thing to make sure is that the pose we are sending to the reach policy is also in distribution. We make sure of this by having a check at the policy level where before any inference, we check if the pose is in distribution. If you see a log message like the one shown below, then the pose is not in distribution and the reach policy is unlikely to work.

[WARNING] [observation_encoder_node]: target position out of distribution

Conclusion

So in essence, the user must make sure to do the following things:

• Ensure the robot is not in an OOD start state

• Ensure the pose we are sending to the reach policy is in distribution

If you have verified all of the above, then the reach policy is likely to work.