Tutorial for Visual Odometry Using multiple HAWK Cameras

Overview

This tutorial walks you through setting up Multi-camera odometry based on Isaac ROS Visual SLAM with a Multiple Hawk cameras mounted on Nova Carter

Tutorial Walkthrough - Multi-camera Visual Odometry Execution

Live on Nova CarterOn a ROSbag

1. Complete the Nova setup tutorial.

2. Complete the Robot teleoperation tutorial with the required packages installed from the apt repository or built from source.

3. [Terminal 1] Clone the Isaac ROS Examples repository:

cd ${ISAAC_ROS_WS}/src
git clone -b release-3.1 https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_examples.git isaac_ros_examples

4. [Terminal 1] Open a new terminal and run the Docker container:

cd ${ISAAC_ROS_WS}/src/isaac_ros_common
./scripts/run_dev.sh

5. [Terminal 1] Inside the running container, build and source the workspace:

cd ${ISAAC_ROS_WS}
colcon build --symlink-install \
    --packages-up-to nova_carter_bringup isaac_ros_multicamera_vo \
    --packages-skip isaac_ros_ess_models_install isaac_ros_peoplesemseg_models_install
source install/setup.bash

6. [Terminal 1] Run the launch file, which launches the multi-camera visual odometry and wait for 10 seconds:

ros2 launch isaac_ros_multicamera_vo isaac_ros_visual_slam_multihawk.launch.py use_rosbag:=False

7. [Terminal 2] Attach another terminal to the running container to execute robot teleoperation:

cd ${ISAAC_ROS_WS}/src/isaac_ros_common
./scripts/run_dev.sh

8. [Terminal 2] Run the robot teleoperation with all sensors disabled:

source ${ISAAC_ROS_WS}/install/setup.bash
ros2 launch nova_carter_bringup teleop.launch.py \
    enable_wheel_odometry:=False \
    enabled_stereo_cameras:=none \
    enabled_fisheye_cameras:=none \
    enabled_2d_lidars:=none \
    enable_3d_lidar:=False \
    run_foxglove:=False

Tutorial Walkthrough - Visualizing the Outputs

Note

Images and landmarks visualization may impact the performance of Visual Odometry. Please use visualization for debugging and demonstration purposes only. To enable landmark visualization, set the following parameters to True in the launch file: enable_slam_visualization, enable_landmarks_view, enable_observations_view

Visualization with Foxglove Studio

Note

These examples will stream the camera images in their full resolution to Foxglove Studio. This requires a substantial amount of bandwidth and is only done here for exemplary purposes. Most likely the image stream will be fairly choppy due to the large bandwidth. For a better visualization experience, we recommend only visualizing resized images or H.264 streams.

1. Complete the foxglove setup guide.

2. In Foxglove Studio open the foxglove_layout_hawk.json layout file, which can be found in the isaac_ros_multicamera_vo repository.

3. Validate that you can see a visualization of the Nova Carter robot, images of the front and right stereo cameras, image landmarks and odometry path. You should expect a visualization similar to the following:

Visualization with RViz2

1. [Terminal 3] Attach another terminal to the running container to execute RViz2:

cd ${ISAAC_ROS_WS}/src/isaac_ros_common
./scripts/run_dev.sh

2. [Terminal 3] Open RViz2 from the new terminal attached to the running docker container:

source ${ISAAC_ROS_WS}/install/setup.bash
rviz2 -d ${ISAAC_ROS_WS}/src/isaac_ros_examples/isaac_ros_multicamera_vo/rviz/hawk_multicamera_vo.rviz

3. Make sure you see the visualization of the Nova Carter robot’s frame tree, images of the front and right stereo camera, image landmarks and odometry path. You should expect a visualization similar to the following: