Isaac ROS Nvblox
Nvblox ROS 2 integration for local 3D scene reconstruction and mapping.
Isaac ROS Nvblox contains ROS 2 packages for 3D reconstruction and cost
maps for navigation.
isaac_ros_nvblox processes depth and pose to
reconstruct a 3D scene in real-time and outputs a 2D costmap for
Nav2. The costmap is
used in planning during navigation as a vision-based solution to avoid
isaac_ros_nvblox is designed to work with depth-cameras and/or 3D LiDAR.
The package uses GPU acceleration to compute a 3D reconstruction and 2D costmaps using
nvblox, the underlying
framework-independent C++ library.
Above is a typical graph that uses
Nvblox takes a depth image, a color image, and a pose as input, with
which it computes a 3D scene reconstruction on the GPU. In this graph
the pose is computed using
visual_slam, or some other pose estimation
node. The reconstruction
is sliced into an output cost map which is provided through a cost map plugin
An optional colorized 3D reconstruction is delivered into
using the mesh visualization plugin. Nvblox streams mesh updates
to RViz to update the reconstruction in real-time as it is built.
isaac_ros_nvblox offers several modes of operation. In its default mode
the environment is assumed to be static. Two additional modes of operation are provided
to support mapping scenes which contain dynamic elements: human reconstruction, for
mapping scenes containing humans, and dynamic reconstruction, for mapping
scenes containing more general dynamic objects.
The graph above shows
isaac_ros_nvblox operating in human reconstruction
mode. The color image corresponding to the depth image is processed with
the PeopleSemSegNet DNN model to estimate a segmentation mask for
persons in the color image. Nvblox uses this mask to separate reconstructed persons into a
separate humans-only part of the reconstruction. The Technical Details
provide more information on these three types of mapping.
The following tables provides timings for various functions of nvblox core on various platforms.
|Voxel Size (m)
|x86_64 w/ 4090 Ti (Desktop)
|x86_64 w/ RTX3000 Ti (Laptop)
This package is designed and tested to be compatible with ROS 2 Humble running on Jetson or an x86_64 system with an NVIDIA GPU.
Versions of ROS 2 earlier than Humble are not supported. This package depends on specific ROS 2 implementation features that were only introduced beginning with the Humble release.
For best performance, ensure that power settings are configured appropriately.
To simplify development, we strongly recommend leveraging the Isaac ROS Dev Docker images by following these steps. This will streamline your development environment setup with the correct versions of dependencies on both Jetson and x86_64 platforms.
All Isaac ROS Quickstarts, tutorials, and examples have been designed with the Isaac ROS Docker images as a prerequisite.
Customize your Dev Environment
To customize your development environment, reference this guide.
General dynamic reconstruction.
Human reconstruction and new weighting functions.
Updated OSS licensing.
Update to be compatible with JetPack 5.0.2. Serialization of Nvblox maps to file. Support for 3D LIDAR input and performance improvements.
Support for ROS 2 Humble and miscellaneous bug fixes.