`isaac_ros_teleop`#

Source code available on GitHub.

Quickstart#

Prerequisites#

Note

This quickstart has been tested and qualified on Jetson AGX Thor and x86_64.

PICO 4 Ultra headset (if no XR headset is available, the emulator provided by Isaac Teleop Core can be used instead)

Set Up Development Environment#

Set up your development environment by following the instructions in getting started.
(Optional) Install dependencies for any sensors you want to use by following the sensor-specific guides.

Note

We strongly recommend installing all sensor dependencies before starting any quickstarts. Some sensor dependencies require restarting the development environment during installation, which will interrupt the quickstart process.

Build `isaac_ros_teleop`#

If using the Isaac ROS Environment in Docker mode, open a new terminal and mount ~/.cloudxr:

mkdir -p ~/.cloudxr && grep -qxF -- '-v `realpath ~/.cloudxr`:/home/admin/.cloudxr' ~/.isaac_ros_dev-dockerargs 2>/dev/null || echo "-v `realpath ~/.cloudxr`:/home/admin/.cloudxr" >> ~/.isaac_ros_dev-dockerargs

Install and build isaac_ros_teleop:

Activate the Isaac ROS environment:
```
isaac-ros activate
```

Install the prebuilt Debian package:

sudo apt-get update

sudo apt-get install -y ros-jazzy-isaac-ros-teleop

Install Git LFS:

sudo apt-get install -y git-lfs && git lfs install

Clone this repository under ${ISAAC_ROS_WS}/src:

cd ${ISAAC_ROS_WS}/src && \
   git clone --recurse-submodules -b release-4.4 https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_teleop.git isaac_ros_teleop

Activate the Isaac ROS environment:
```
isaac-ros activate
```

Use rosdep to install the package’s dependencies:

sudo apt-get update

rosdep update && rosdep install --from-paths ${ISAAC_ROS_WS}/src/isaac_ros_teleop/isaac_ros_teleop --ignore-src -y

Build the package from source:

cd ${ISAAC_ROS_WS}/ && \
   colcon build --symlink-install --packages-up-to isaac_ros_teleop --base-paths ${ISAAC_ROS_WS}/src/isaac_ros_teleop/isaac_ros_teleop

Source the ROS workspace:

Note

Make sure to repeat this step in every terminal created inside the Isaac ROS environment. Because this package was built from source, the enclosing workspace must be sourced for ROS to be able to find the package’s contents.
```
source install/setup.bash
```

Run CloudXR#

If not already done, clone isaac_ros_teleop:

cd ${ISAAC_ROS_WS}/src && \
   git clone --recurse-submodules -b release-4.4 https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_teleop.git isaac_ros_teleop

In a new terminal, run CloudXR Runtime via the CloudXR Docker Container:

cd ${ISAAC_ROS_WS}/src/isaac_ros_teleop/isaac_teleop_core/IsaacTeleop && \
   ./scripts/run_cloudxr_via_docker.sh

Connect the XR headset to the teleop server. Follow the headset connection guide.

Note

If you are running this on Thor, make sure to set the Video Codec to H.264, otherwise the headset will fail to connect.

Important

The world frame of the headset is defined as the position of the headset and controllers at the moment of connection. To reset the world frame, disconnect and reconnect the headset while stationary. If teleoperation with a robot is performed, stand still and face the robot before connecting to establish a consistent world frame.

Run Launch File#

Now, in the original terminal, run the following launch file to spin up a demo of this package:
```
ros2 launch isaac_ros_teleop isaac_ros_teleop.launch.py
```

Visualize Results#

Open a new terminal and activate the Isaac ROS environment:
```
isaac-ros activate
```
Observe the controller output /xr_teleop/ee_poses on a separate terminal with the command:
```
ros2 topic echo /xr_teleop/ee_poses
```

Controller Reference#

The PICO 4 Ultra headset includes two handheld controllers. The following table summarizes what each input does during teleoperation:

Input	Action
Left joystick	Move the robot: up = forward, down = backward, left = strafe left, right = strafe right
Right joystick — left / right	Rotate the robot in place (yaw)
Controller motion (6-DOF)	The end-effector pose tracks the physical controller; moving and rotating the controller moves the robot’s hand correspondingly
Triggers (each controller has two)	Open and close the finger joints of the tri-finger hand

Troubleshooting#

Isaac ROS Troubleshooting#

For solutions to problems with Isaac ROS, see troubleshooting.

API#

Usage#

ros2 launch isaac_ros_teleop isaac_ros_teleop.launch.py

ROS Parameters#

Launch Argument	Type	Default	Description
`ee_pose_topic`	`string`	`xr_teleop/ee_poses`	Topic name for published end-effector poses (`geometry_msgs/PoseArray`)
`root_twist_topic`	`string`	`xr_teleop/root_twist`	Topic name for published root velocity command (`geometry_msgs/TwistStamped`)
`root_pose_topic`	`string`	`xr_teleop/root_pose`	Topic name for published root pose command (`geometry_msgs/PoseStamped`)
`finger_joints_topic`	`string`	`xr_teleop/finger_joints`	Topic name for the retargeted finger joints (`sensor_msgs/JointState`)
`rate_hz`	`double`	`60.0`	The publishing rate in Hz
`world_frame`	`string`	`world`	World frame for message headers and TF parent frame
`right_wrist_frame`	`string`	`right_wrist`	TF child frame name for the right wrist
`left_wrist_frame`	`string`	`left_wrist`	TF child frame name for the left wrist

ROS Topics Published#

ROS Topic	Interface	Description
`ee_pose_topic`	geometry_msgs/PoseArray	The poses of the wrists/end-effector; given in order of right wrist and then left wrist.
`root_twist_topic`	geometry_msgs/TwistStamped	The root twist command.
`root_pose_topic`	geometry_msgs/PoseStamped	The root pose command.
`finger_joints_topic`	sensor_msgs/JointState	The retargeted finger joint angles for the robot.
`tf`	tf2_msgs/TFMessage	Poses of the wrists w.r.t to the world frame `world_frame`.

isaac_ros_teleop#