Isaac ROS Freespace Segmentation

Isaac ROS Freespace Segmentation Sample Output

Overview

Isaac ROS Freespace Segmentation contains an ROS 2 package to produce occupancy grids for navigation. By processing a freespace segmentation mask with the pose of the robot relative to the ground, Bi3D Freespace produces an occupancy grid for Nav2, which is used to avoid obstacles during navigation. This package is GPU accelerated to provide real-time, low latency results in a robotics application. Bi3D Freespace provides an additional occupancy grid source for mobile robots (ground based).

Isaac ROS Freespace Segmentation Sample Output

isaac_ros_bi3d is used in a graph of nodes to provide a freespace segmentation mask as one output from a time-synchronized input left and right stereo image pair. The freespace mask is used by isaac_ros_bi3d_freespace with TF pose of the camera relative to the ground to compute planar freespace into an occupancy grid as input to Nav2.

There are multiple methods to predict the occupancy grid as an input to navigation. None of these methods are perfect; each has limitations on the accuracy of its estimate from the sensor providing measured observations. Each sensor has a unique field of view, range to provide its measured view of the world, and corresponding areas it does not measure. Bi3D Freespace provides a diverse approach to identifying obstacles from freespace. Stereo camera input used for this function is diverse relative to lidar, and has a better vertical field of view than most lidar units, allowing for perception of low lying obstacles that lidar can miss. Bi3D Freespace provides a robust, vision-based complement to lidar occupancy scanning.

Quickstarts

Isaac ROS NITROS Acceleration

This package is powered by NVIDIA Isaac Transport for ROS (NITROS), which leverages type adaptation and negotiation to optimize message formats and dramatically accelerate communication between participating nodes.

Packages

Supported Platforms

This package is designed and tested to be compatible with ROS 2 Humble running on Jetson or an x86_64 system with an NVIDIA GPU.

Note

Versions of ROS 2 other than Humble are not supported. This package depends on specific ROS 2 implementation features that were introduced beginning with the Humble release. ROS 2 versions after Humble have not yet been tested.

Platform

Hardware

Software

Notes

Jetson

Jetson Orin

JetPack 6.0

For best performance, ensure that power settings are configured appropriately.


Jetson Orin Nano 4GB may not have enough memory to run many of the Isaac ROS packages and is not recommended.

x86_64

Ampere or higher NVIDIA GPU Architecture with 8 GB RAM or higher

Ubuntu 22.04+

CUDA 12.2+

Docker

To simplify development, we strongly recommend leveraging the Isaac ROS Dev Docker images by following these steps. This will streamline your development environment setup with the correct versions of dependencies on both Jetson and x86_64 platforms.

Note

All Isaac ROS Quickstarts, tutorials, and examples have been designed with the Isaac ROS Docker images as a prerequisite.

Customize your Dev Environment

To customize your development environment, reference this guide.

Updates

Date

Changes

2024-09-26

Update for ZED compatibility

2024-05-30

Update to be compatible with JetPack 6.0

2023-10-18

Updated for Isaac ROS 2.0.0

2023-05-25

Performance improvements

2023-04-05

Initial release