Tutorial: Teleoperate the Robot with a Joystick

This demo will enable you to remote control the robot with a joystick. Additionally, you will be able to visualize the outputs of the various sensors on the robot with Foxglove.

For this tutorial, it is assumed that you have successfully setup your Nova Carter robot using the setup instructions here and you are familiar with the Isaac ROS development environment.

Instructions

1. SSH into the robot (instructions).

2. Make sure you have successfully connected the PS5 joystick to the robot (instructions).

3. Build/install the required packages:

Docker ImageBinary PackageBuild from Source

1. Pull the Docker image:

docker pull nvcr.io/nvidia/isaac/nova_carter_bringup:release_3.1-aarch64

2. Run the Docker image:

docker run --privileged --network host \
    -v /dev/*:/dev/* \
    -v /tmp/argus_socket:/tmp/argus_socket \
    -v /etc/nova:/etc/nova \
    nvcr.io/nvidia/isaac/nova_carter_bringup:release_3.1-aarch64 \
    ros2 launch nova_carter_bringup teleop.launch.py

4. You are now able to remote control the robot with the gamepad. You can use the left joystick to control the linear velocity and the right joystick to control the angular velocity of the robot. Please note that you have to press L1 to enable the remote control. You can enable turbo mode by pressing R1.

   Follow the next steps to additionally visualize the sensor outputs in Foxglove.

Visualizing the Outputs

1. Make sure you complete Visualization Setup.

2. Open the Foxglove studio on your remote machine. In Foxglove, open the nova_carter_teleop.json layout file downloaded in the previous step.

3. Validate that you can see a visualization of the Nova Carter robot, an image of the front stereo camera and the scan lines of the front and back 2D lidar scanners. You should expect a visualization similar to the following:

You can also visualize different sensors by enabling them in the left-hand panel.

Note

This example will stream the camera images in their full resolution to Foxglove. This requires a substantial amount of bandwidth and is only done here for exemplary purposes. Most likely the image stream will be fairly choppy due to the large bandwidth. For a better visualization experience, we recommend only visualizing resized images or h264 streams.

Note

Streaming higher bandwidth data to Foxglove causes higher latency. You can choose an appropriate value for the send_buffer_limit parameter in Foxglove bridge to control latency vs. high bandwidth data visualization.

Note

The fisheye cameras use the equidistant distortion model which is not yet supported in Foxglove. For this reason, it is not possible to visualize a fisheye camera in the 3D panel in Foxglove. You can however add an additional Image panel and use this to visualize the fisheye camera. In the Image panel’s settings set Topic to the fisheye camera’s image topic and set Calibration to None.

Enabling Different Sensors

Per default, the launch file teleop.launch.py will enable the Segway base, the front, left and right stereo cameras, the front and back 2D lidars and the 3D lidar. You can however pass additional launch arguments to enable different sensors.

To see all available launch arguments run

ros2 launch nova_carter_bringup teleop.launch.py --show-args

To enable all sensors use the following launch command

ros2 launch nova_carter_bringup teleop.launch.py \
    enable_3d_lidar:=True \
    enabled_2d_lidars:=front_2d_lidar,back_2d_lidar \
    enabled_stereo_cameras:=front_stereo_camera,back_stereo_camera,left_stereo_camera,right_stereo_camera \
    enabled_fisheye_cameras:=front_fisheye_camera,back_fisheye_camera,left_fisheye_camera,right_fisheye_camera

To disable all sensors and only run the Segway base use the following launch command

ros2 launch nova_carter_bringup teleop.launch.py \
    enable_3d_lidar:=False \
    enabled_2d_lidars:=none \
    enabled_stereo_cameras:=none \
    enabled_fisheye_cameras:=none

You may also change the launch arguments to selectively dis-/enable sensors.