isaac_ros_centerpose

Source code on GitHub.

Quickstart

1. Complete steps 1-4 of the quickstart here

2. Install this package’s dependencies.

   sudo apt-get install -y ros-humble-isaac-ros-centerpose ros-humble-isaac-ros-triton ros-humble-isaac-ros-dnn-image-encoder
   

3. Create a models repository with version 1:

   mkdir -p /tmp/models/centerpose_shoe/1
   

4. Copy the test model located in isaac_ros_centerpose into /tmp/models/centerpose_shoe/1. Make sure it’s called model.onnx:

   cp /workspaces/isaac_ros-dev/src/isaac_ros_pose_estimation/isaac_ros_centerpose/test/models/centerpose_shoe.onnx /tmp/models/centerpose_shoe/1/model.onnx
   

5. Create a configuration file for this model at path /tmp/models/centerpose_shoe/config.pbtxt. Note that name has to be the same as the model repository name. Take a look at the example at isaac_ros_centerpose/test/models/centerpose_shoe/config.pbtxt and copy that file to /tmp/models/centerpose_shoe/config.pbtxt.

   cp /workspaces/isaac_ros-dev/src/isaac_ros_pose_estimation/isaac_ros_centerpose/test/models/centerpose_shoe/config.pbtxt /tmp/models/centerpose_shoe/config.pbtxt
   

6. To get a TensorRT engine plan file with Triton, export the ONNX model into an TensorRT engine plan file using the builtin TensorRT converter trtexec:

   /usr/src/tensorrt/bin/trtexec --onnx=/tmp/models/centerpose_shoe/1/model.onnx --saveEngine=/tmp/models/centerpose_shoe/1/model.plan
   

7. Start isaac_ros_centerpose using the launch file:

   ros2 launch isaac_ros_centerpose isaac_ros_centerpose_triton.launch.py model_name:=centerpose_shoe model_repository_paths:=['/tmp/models']
   

   Then open another terminal, and enter the Docker container again:

   cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
     ./scripts/run_dev.sh
   

   Then, play the ROS bag:

   ros2 bag play -l src/isaac_ros_pose_estimation/resources/rosbags/centerpose_rosbag/
   

8. Open another terminal window and attach to the same container. You should be able to get the poses of the objects in the images through ros2 topic echo:

   In a third terminal, enter the Docker container again:

   cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
     ./scripts/run_dev.sh
   

   ros2 topic echo /centerpose/detections
   

   Then launch rqt_image_view:

   ros2 run rqt_image_view rqt_image_view
   

   Then inside the rqt_image_view GUI, change the topic to /centerpose/image_visualized to view the detections projected back onto the input image.

Note

The launch file launches a visualization node for visualization purposes.

There also is an analogous launch file called isaac_ros_centerpose_tensor_rt.launch.py that can be used if using TensorRT is desired instead of Triton.

Warning

CenterPose only produces poses up to a scale. There are a few ways to obtain the scale factor. The original paper simply measured the height of each object. This scale factor can be set using the cuboid_scaling_factor.

Use Different Models

NGC has CenterPose class models that can detect other objects. Check them out here.

Troubleshooting

Isaac ROS Troubleshooting

For solutions to problems with Isaac ROS, please check here.

Deep Learning Troubleshooting

For solutions to problems with using DNN models, please check here.

API

Usage

Two launch files are provided to use this package. The first launch file launches isaac_ros_tensor_rt, whereas the other one uses isaac_ros_triton, along with the necessary components to perform encoding on images and decoding of the CenterPose network’s output.

Warning

For your specific application, these launch files may need to be modified. Please consult the available components to see the configurable parameters.

Launch File	Components Used
isaac_ros_centerpose_tensor_rt.launch.py	DnnImageEncoderNode, TensorRTNode, CenterPoseDecoderNode, CenterPoseVisualizerNode
isaac_ros_centerpose_triton.launch.py	DnnImageEncoderNode, TritonNode, CenterPoseDecoderNode, CenterPoseVisualizerNode

CenterPoseDecoderNode

ROS Parameters

ROS Parameter	Type	Default	Description
output_field_size	int list	[128, 128]	An array of two integers that represent the size of the 2D keypoint decoding from the network output
cuboid_scaling_factor	float	1.0	This parameter is used to scale the cuboid used for calculating the size of the objects detected.
score_threshold	float	0.3	The threshold for scores values to discard. Any score less than this value will be discarded.
object_name	string	shoe	The name of the category instance / object detected.

ROS Topics Subscribed

ROS Topic	Interface	Description
tensor_sub	isaac_ros_tensor_list_interfaces/TensorList	The TensorList that contains the outputs of the CenterPose network.
camera_info	sensor_msgs/CameraInfo	The CameraInfo of the original, undistorted image.

ROS Topics Published

ROS Topic	Interface	Description
centerpose/detections	vision_msgs/Detection3DArray	A Detection3DArray representing the poses of objects detected by the CenterPose network and interpreted by the CenterPose decoder node.

CenterPoseVisualizerNode

ROS Parameters

ROS Parameter	Type	Default	Description
show_axes	bool	true	Whether to draw the axes of the detected pose or not.
bounding_box_color	int32_t	0x000000ff	The color of the bounding box drawn. Only the last 24 bits are used to draw the color.

ROS Topics Subscribed

ROS Topic	Interface	Description
image	sensor_msgs/Image	The original, undistorted image.
camera_info	sensor_msgs/CameraInfo	The CameraInfo of the original, undistorted image.
centerpose/detections	vision_msgs/Detection3DArray	The detections made the CenterPose decoder node.

ROS Topics Published

ROS Topic	Interface	Description
centerpose/image_visualized	sensor_msgs/Image	An image containing the detection’s bounding box reprojected onto the image and, optionally, the axes of the detected objects.