Isaac ROS Pose Estimation
Isaac ROS Pose Estimation
contains ROS 2 packages to predict the pose of an object.
isaac_ros_dope estimates the object’s pose using the 3D
bounding cuboid dimensions of a known object in an input image.
isaac_ros_centerpose estimates the object’s pose using the 3D
bounding cuboid dimensions of unknown object instances in a known
category of objects from an input image.
isaac_ros_centerpose use GPU acceleration for DNN inference to
estimate the pose of an object. The output prediction can be used by
perception functions when fusing with the corresponding depth to provide
the 3D pose of an object and distance for navigation or manipulation.
isaac_ros_dope is used in a graph of nodes to estimate the pose of a
known object with 3D bounding cuboid dimensions. To produce the
estimate, a DOPE (Deep
Object Pose Estimation) pre-trained model is required. Input images may
need to be cropped and resized to maintain the aspect ratio and match
the input resolution of DOPE. After DNN inference has produced an estimate, the
DNN decoder will use the specified object type, along with the belief
maps produced by model inference, to output object poses.
NVLabs has provided a DOPE pre-trained model using the
HOPE dataset. HOPE stands
Household Objects for Pose Estimation. HOPE is a research-oriented
dataset that uses toy grocery objects and 3D textured meshes of the objects
for training on synthetic data. To use DOPE for other objects that are
relevant to your application, the model needs to be trained with another
dataset targeting these objects. For example, DOPE has been trained to
detect dollies for use with a mobile robot that navigates under, lifts,
and moves that type of dolly. To train your own DOPE model, please refer to the
Training your Own DOPE Model Tutorial.
isaac_ros_centerpose has similarities to
isaac_ros_dope in that
both estimate an object pose; however,
additional functionality. The
CenterPose DNN performs
object detection on the image, generates 2D keypoints for the object,
estimates the 6-DoF pose up to a scale, and regresses relative 3D bounding cuboid
dimensions. This is performed on a known object class without knowing
the instance-for example, a CenterPose model can detect a chair without having trained on
images of that specific chair.
Pose estimation is a compute-intensive task and therefore not performed at the frame rate of an input camera. To make efficient use of resources, object pose is estimated for a single frame and used as an input to navigation. Additional object pose estimates are computed to further refine navigation in progress at a lower frequency than the input rate of a typical camera.
Packages in this repository rely on accelerated DNN model inference
using Triton or
TensorRT from Isaac ROS DNN Inference.
For preprocessing, packages in this rely on the
Isaac ROS DNN Image Encoder,
which can also be found at Isaac ROS DNN Inference.
This package is designed and tested to be compatible with ROS 2 Humble running on Jetson or an x86_64 system with an NVIDIA GPU.
Versions of ROS 2 earlier than Humble are not supported. This package depends on specific ROS 2 implementation features that were only introduced beginning with the Humble release.
For best performance, ensure that power settings are configured appropriately.
To simplify development, we strongly recommend leveraging the Isaac ROS Dev Docker images by following these steps. This will streamline your development environment setup with the correct versions of dependencies on both Jetson and x86_64 platforms.
All Isaac ROS Quickstarts, tutorials, and examples have been designed with the Isaac ROS Docker images as a prerequisite.
Customize your Dev Environment
To customize your development environment, reference this guide.
Adding NITROS CenterPose decoder
Source available GXF extensions
Update to use NITROS for improved performance and to be compatible with JetPack 5.0.2
Refactored README, updated launch file & added