isaac_manipulator_servers
Source code on GitHub.
Overview
isaac_manipulator_servers
package contains the implementation of the following servers:
Object Detection Server: For detecting objects in the scene using an object detection network, such as RT-DETR.
Pose Estimation Server: For estimating the pose of the object in the scene using a pose estimation network, such as FoundationPose.
Object Info Server:, which wraps the aforementioned servers. This provides a common interface for getting object information, such as the object’s pose or the 2D bounding box of the object within an image.
From a high level, a Object Detection Server and Pose Estimation Server continuously subscribes to the inputs and stores the latest received input. For example, the Object Detection Server continuously subscribes to images and stores the latest image. Once the action is triggered, it will publish the latest image that it received to the specified topic. It then expects a pipeline, such as the RT-DETR pipeline, to subscribe this image and process it to produce a detection. This detection should then be published so that the Object Detection Server can receive it and pass it forward to the subsequent part of the pipeline.
API
ObjectDetectionServer
ROS Parameters
ROS Parameter |
Type |
Default |
Description |
---|---|---|---|
|
|
|
The name of the object detection action. |
|
|
|
The topic that the server will subscribe to for input images. |
|
|
|
When executed, the server will publish messages to this topic. |
|
|
|
The server receives the actual object detection through this topic. |
|
|
|
The server publishes the received input detections to this topic. |
|
|
|
Subscription QoS value. |
|
|
|
Publisher QoS value. |
ROS Topics Subscribed
ROS Topic |
Interface |
Description |
---|---|---|
|
The input image that will be used for object detection. The latest image is stored, and will be sent through the |
|
|
The resultant detections coming from a object detection model. |
ROS Topics Published
ROS Topic |
Interface |
Description |
---|---|---|
|
The image that is forwarded to object detection model |
|
|
Detections forwarded from object detection model. |
ROS Actions Advertised
ROS Action |
Interface |
Description |
---|---|---|
|
Action server for object detection models. |
DopeServer
Note
At the present, this node is not used.
ROS Parameters
ROS Parameter |
Type |
Default |
Description |
---|---|---|---|
|
|
|
The name of the DOPE action server. |
|
|
|
The topic that the server will subscribe to for input images. |
|
|
|
When executed, the server will publish an image to this topic. |
|
|
|
The topic that the server will subscribe to for input camera info messages. |
|
|
|
When executed, the server will publish a camera info to this topic. |
|
|
|
The server receives the actual poses through this topic. |
|
|
|
The server publishes the received input poses to this topic. |
|
|
|
The server receives the actual 2D detections through this topic. |
|
|
|
The server publishes the received input 2D detections to this topic. |
|
|
|
Whether or not the server should expect to receive 2D detections or not. |
|
|
|
Subscription QoS value. |
|
|
|
Publisher QoS value. |
ROS Topics Subscribed
ROS Topic |
Interface |
Description |
---|---|---|
|
The input image that is used for pose estimation. The latest image is stored. |
|
|
CameraInfo messages for input images. The latest CameraInfo is stored. |
|
|
The resultant pose coming from the model. |
|
|
The resultant 2D detections coming from the model. |
ROS Topics Published
ROS Topic |
Interface |
Description |
---|---|---|
|
The image that is forwarded to the model |
|
|
CameraInfo messages which are forwarded to the model. |
|
|
Pose estimation forwarded from the model. |
|
|
2D detections forwarded from the model. |
ROS Actions Advertised
ROS Action |
Interface |
Description |
---|---|---|
|
Action server for DOPE DNN model. |
FoundationPoseServer
ROS Parameters
ROS Parameter |
Type |
Default |
Description |
---|---|---|---|
|
|
|
The name of the foundation pose action. |
|
|
|
The topic that the server will subscribe to for input images. |
|
|
|
When executed, the server will publish images to this topic. |
|
|
|
The topic that the server will subscribe to for input camera info messages. |
|
|
|
When executed, the server will publish camera info messages to this topic. |
|
|
|
The topic that the server will subscribe to for input depth images. |
|
|
|
When executed, the server will publish depth images to this topic. |
|
|
|
The server receives the actual pose estimation through this topic. |
|
|
|
The server publishes the received pose estimations to this topic. |
|
|
|
The server publishes the bounding box that it receives as part of the goal to this topic. |
|
|
|
Subscription QoS value. |
|
|
|
Publisher QoS value. |
ROS Topics Subscribed
ROS Topic |
Interface |
Description |
---|---|---|
|
The input image that is used for pose estimation. The latest image is stored. |
|
|
CameraInfo messages for input images. The latest camera info is stored. |
|
|
The input depth image that is used for pose estimation. The latest depth image is stored. |
|
|
The resultant pose estimation coming from the model. |
ROS Topics Published
ROS Topic |
Interface |
Description |
---|---|---|
|
The image that is forwarded to the model. |
|
|
CameraInfo messages which are forwarded to the model. |
|
|
The depth image that is forwarded to the model |
|
|
Bounding box data forwarded to the model. |
|
|
Pose estimation forwarded from the model. |
ROS Actions Advertised
ROS Action |
Interface |
Description |
---|---|---|
|
isaac_manipulator_interfaces::action::EstimatePoseFoundationPose |
Action server for Foundation Pose DNN model. |
ObjectInfoServer
ROS Parameters
ROS Parameter |
Type |
Default |
Description |
---|---|---|---|
|
|
|
Object detection backend to use. The options currently supported are |
|
|
|
Pose estimation backend to use. The options currently supported are |
ROS Actions Advertised
ROS Action |
Interface |
Description |
---|---|---|
|
Wrapper action server for object detection server. |
|
|
Wrapper action server for pose estimation server. |
ROS Services Advertised
ROS Service |
Interface |
Description |
---|---|---|
|
Service to clear the cached object info like 2D detections and pose estimation. |