Isaac ROS Visual SLAM

Webinar Available

Learn how to use this package by watching our on-demand webinar: Pinpoint, 250 fps, ROS 2 Localization with vSLAM on Jetson

Overview

Isaac ROS Visual SLAM provides a high-performance, best-in-class ROS 2 package for VSLAM (visual simultaneous localization and mapping). This package uses one or more stereo cameras and optionally an IMU to estimate odometry as an input to navigation. It is GPU accelerated to provide real-time, low-latency results in a robotics application. VSLAM provides an additional odometry source for mobile robots (ground based) and can be the primary odometry source for drones.

VSLAM provides a method for visually estimating the position of a robot relative to its start position, known as VO (visual odometry). This is particularly useful in environments where GPS is not available (such as indoors) or intermittent (such as urban locations with structures blocking line of sight to GPS satellites). This method is designed to use multiple stereo camera frames and an IMU (inertial measurement unit) as input. It uses stereo image pairs to find matching key points. Using the baseline between the camera pairs, it can estimate the distance to the key point. Using consecutive images, VSLAM can track the motion of key points to estimate the 3D motion of the camera-which is then used to compute odometry as an output to navigation. Compared to the classic approach to VSLAM, this method uses GPU acceleration to find and match more key points in real-time, with fine tuning to minimize overall reprojection error.

Key points depend on distinctive features in the images that can be repeatedly detected with changes in size, orientation, perspective, lighting, and image noise. In some instances, the number of key points may be limited or entirely absent; for example, if the camera field of view is only looking at a large solid colored wall, no key points may be detected. If there are insufficient key points, this module uses motion sensed with the IMU to provide a sensor for motion, which, when measured, can provide an estimate for odometry. This method, known as VIO (visual-inertial odometry), improves estimation performance when there is a lack of distinctive features in the scene to track motion visually.

SLAM (simultaneous localization and mapping) is built on top of VIO, creating a map of key points that can be used to determine if an area is previously seen. When VSLAM determines that an area is previously seen, it reduces uncertainty in the map estimate, which is known as loop closure. VSLAM uses a statistical approach to loop closure that is more compute efficient to provide a real time solution, improving convergence in loop closure.

There are multiple methods for estimating odometry as an input to navigation. None of these methods are perfect; each has limitations because of systematic flaws in the sensor providing measured observations, such as missing LIDAR returns absorbed by black surfaces, inaccurate wheel ticks when the wheel slips on the ground, or a lack of distinctive features in a scene limiting key points in a camera image. A practical approach to tracking odometry is to use multiple sensors with diverse methods so that systemic issues with one method can be compensated for by another method. With three separate estimates of odometry, failures in a single method can be detected, allowing for fusion of the multiple methods into a single higher quality result. VSLAM provides a vision- and IMU-based solution to estimating odometry that is different from the common practice of using LIDAR and wheel odometry. VSLAM can even be used to improve diversity, with multiple stereo cameras positioned in different directions to provide multiple, concurrent visual estimates of odometry.

To learn more about VSLAM, refer to the cuVSLAM SLAM documentation.

Quickstarts

• Isaac ROS Visual SLAM

Accuracy

VSLAM is a best-in-class package with the lowest translation and rotational error as measured on KITTI Visual Odometry / SLAM Evaluation 2012 for real-time applications.

Method	Runtime	Translation	Rotation	Platform
VSLAM	0.007s	0.94%	0.0019 deg/m	Jetson AGX Xavier aarch64
ORB-SLAM2	0.06s	1.15%	0.0027 deg/m	2 cores @ >3.5 GHz x86_64

In addition to results from standard benchmarks, we test loop closure for VSLAM on sequences of over 1000 meters, with coverage for indoor and outdoor scenes.

Packages

• isaac_ros_visual_slam
  • Quickstart
  • Try More Examples
  • Coordinate Frames
  • Troubleshooting
  • API
• isaac_ros_visual_slam_interfaces

Camera System Requirements

The camera system providing data to this package must adhere to the specifications outlined below:

Specification	Required Specification
Minimum target imager framerate	30 Hertz
Maximum permissible jitter in imager framerate	+/- 2 milliseconds
Maximum expected offset between imagers within stereo camera	+/- 100 microseconds
Maximum expected offset between imagers across stereo camera	+/- 100 microseconds

Supported Platforms

This package is designed and tested to be compatible with ROS 2 Humble running on Jetson or an x86_64 system with an NVIDIA GPU.

Note

Versions of ROS 2 other than Humble are not supported. This package depends on specific ROS 2 implementation features that were introduced beginning with the Humble release. ROS 2 versions after Humble have not yet been tested.

Platform	Hardware	Software	Notes
Jetson	Jetson Orin	JetPack 6.0	For best performance, ensure that power settings are configured appropriately. Jetson Orin Nano 4GB may not have enough memory to run many of the Isaac ROS packages and is not recommended.
x86_64	Ampere or higher NVIDIA GPU Architecture with 8 GB RAM or higher	Ubuntu 22.04+	CUDA 12.2+

Docker

To simplify development, we strongly recommend leveraging the Isaac ROS Dev Docker images by following these steps. This will streamline your development environment setup with the correct versions of dependencies on both Jetson and x86_64 platforms.

Note

All Isaac ROS Quickstarts, tutorials, and examples have been designed with the Isaac ROS Docker images as a prerequisite.

Customize your Dev Environment

To customize your development environment, reference this guide.

Commercial Support & Source Access

For commercial support and access to source code, please contact NVIDIA.

Updates

Date	Changes
2024-09-26	Update for ZED compatibility
2024-05-30	Add support for multi-cam VIO
2023-10-18	Improved stability
2023-05-25	cuVSLAM 11 with improved quality
2023-04-05	Update Elbrus library and performance improvements
2022-10-19	Updated OSS licensing
2022-08-31	Update to be compatible with JetPack 5.0.2
2022-06-30	Support for ROS 2 Humble
2022-03-17	Documentation update for new features
2022-03-11	Renamed to isaac_ros_visual_slam
2021-11-15	Isaac Sim HIL documentation update
2021-10-20	Initial release