isaac_ros_ground_calibration

Source code on GitHub.

Overview

The isaac_ros_ground_calibration package enables calibration of any color or grayscale camera with respect to the ground plane.

Autonomous robotic systems need to perceive obstacles and the environment around them. Ground calibration helps perception modules to identify the ground plane, and differentiate it from objects and other features in the scene.

Ground calibration refers to the 3 Degrees of Freedom (DoF) aligning a sensor or a robot with the ground plane, and includes elevation in the Z axis, as well as pitch and roll rotations.

This package is expected to be used offline and requires a calibration target on the ground. Refer to the Calibration Target section below for printing instructions.

The main inputs of the camera-ground calibration tool are a recording of a scene including one static calibration target placed on flat ground (refer to the Data Recording section below), the measurements of the calibration target in a YAML file, and the topic of the camera observing the target.

The output of this tool is a 3-DoF transformation between a camera in canonical frame (X forward, Y left, Z up) and the ground plane. A sample output is shown below:

Plane calibration results (3-DoF):
plane_T_front_stereo_camera_left: z: 0.339673 m,  roll: -0.006314 rad, pitch: -0.020137 rad

Calibration can be substituted as follows in a URDF file (using nominal values for the remaining non-estimated 3 DoF):
<joint name="front_stereo_camera_left_joint" type="fixed">
   <origin xyz="<nominal_x> <nominal_y> 0.339673" rpy="-0.006314 -0.020137 <nominal_yaw>"/>
   <parent link="base_link"/>
   <child link="front_stereo_camera_left"/>
</joint>

Quickstart

Set Up Development Environment

1. Set up your development environment by following the instructions in getting started.

2. Clone isaac_ros_common under ${ISAAC_ROS_WS}/src.

   cd ${ISAAC_ROS_WS}/src && \
      git clone -b release-3.2 https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common.git isaac_ros_common
   

3. (Optional) Install dependencies for any sensors you want to use by following the sensor-specific guides.

   Note

   We strongly recommend installing all sensor dependencies before starting any quickstarts. Some sensor dependencies require restarting the Isaac ROS Dev container during installation, which will interrupt the quickstart process.

Download Quickstart Assets

1. Download quickstart data from NGC:

   Make sure required libraries are installed.

   sudo apt-get install -y curl jq tar
   

   Then, run these commands to download the asset from NGC:

   NGC_ORG="nvidia"
   NGC_TEAM="isaac"
   PACKAGE_NAME="isaac_ros_ground_calibration"
   NGC_RESOURCE="isaac_ros_ground_calibration_assets"
   NGC_FILENAME="quickstart.tar.gz"
   MAJOR_VERSION=3
   MINOR_VERSION=2
   VERSION_REQ_URL="https://catalog.ngc.nvidia.com/api/resources/versions?orgName=$NGC_ORG&teamName=$NGC_TEAM&name=$NGC_RESOURCE&isPublic=true&pageNumber=0&pageSize=100&sortOrder=CREATED_DATE_DESC"
   AVAILABLE_VERSIONS=$(curl -s \
       -H "Accept: application/json" "$VERSION_REQ_URL")
   LATEST_VERSION_ID=$(echo $AVAILABLE_VERSIONS | jq -r "
       .recipeVersions[]
       | .versionId as \$v
       | \$v | select(test(\"^\\\\d+\\\\.\\\\d+\\\\.\\\\d+$\"))
       | split(\".\") | {major: .[0]|tonumber, minor: .[1]|tonumber, patch: .[2]|tonumber}
       | select(.major == $MAJOR_VERSION and .minor <= $MINOR_VERSION)
       | \$v
       " | sort -V | tail -n 1
   )
   if [ -z "$LATEST_VERSION_ID" ]; then
       echo "No corresponding version found for Isaac ROS $MAJOR_VERSION.$MINOR_VERSION"
       echo "Found versions:"
       echo $AVAILABLE_VERSIONS | jq -r '.recipeVersions[].versionId'
   else
       mkdir -p ${ISAAC_ROS_WS}/isaac_ros_assets && \
       FILE_REQ_URL="https://api.ngc.nvidia.com/v2/resources/$NGC_ORG/$NGC_TEAM/$NGC_RESOURCE/\
   versions/$LATEST_VERSION_ID/files/$NGC_FILENAME" && \
       curl -LO --request GET "${FILE_REQ_URL}" && \
       tar -xf ${NGC_FILENAME} -C ${ISAAC_ROS_WS}/isaac_ros_assets && \
       rm ${NGC_FILENAME}
   fi
   

Build isaac_ros_ground_calibration

Binary PackageBuild from Source

1. Launch the Docker container using the run_dev.sh script:

   cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
   ./scripts/run_dev.sh
   

2. Install the prebuilt Debian package:

   sudo apt-get update
   

   sudo apt update
   sudo apt-get install -y ros-humble-isaac-ros-ground-calibration
   

3. Declare ROS_DOMAIN_ID with the same unique ID (number between 0 and 101) on every bash instance inside the Docker container:

   export ROS_DOMAIN_ID=<unique ID>
   

Run Launch File

1. Continuing inside the Docker container, run the following launch file to spin up a demo of this package:

   ros2 launch isaac_ros_ground_calibration camera_ground_calibration.launch.py \
     rosbag:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_ground_calibration/quickstart_rosbag \
     image_topic:=/front_stereo_camera/left/image_compressed \
     target_file:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_ground_calibration/quickstart_target_info.yaml \
     rectify_width:=1920 rectify_height:=1200
   

   (Optional) Add an output file path to the launch command to keep a copy of the calibration results, using the output argument.

Note

The arguments rectify_width and rectify_height must reflect the image resolution of the camera used, and they only need to be provided if the image stream is not already rectified.

Visualize

1. Open a new terminal inside the Docker container:

   cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
      ./scripts/run_dev.sh
   

2. Declare ROS_DOMAIN_ID with the same unique ID as in the main terminal:

   export ROS_DOMAIN_ID=<unique ID>
   

3. Run RViz or rqt_image_view to visualize the detection and pose estimation of the calibration target. By default, detections will be published in /camera_ground_calibration/target_detection. Images will stop being published after ground calibration succeeds:

   ros2 run rqt_image_view rqt_image_view
   

   

Calibration Target

Supported Patterns

The current version of the camera-ground calibration tool only supports checkerboards.

Note

Checkerboards must be fully on sight of the camera for them to be detected. Partial detections are not supported.

Size and Design

The required size of the calibration target depends on the camera configuration and sensor placement in the robot, including but not limited to distance to the ground, incidence angle, image resolution, and lens choice.

As a general guideline it is recommended to use a calibration target with large square size, and a minimum of 7 rows and 11 columns. Large calibration targets are preferred, as long as the squares that are further away from the camera still appear large in the image and their detection is stable. Hence, the larger the target, the larger the square size needs to be. Moreover, the smaller the image resolution used for target detection, the larger the square size needs to be.

The specifications for the calibration targets used in this tutorial are as follows:

Pattern Type	Board Dimension	Thickness	Square Size	Checkerboard Resolution
Checkerboard Coarse	400x300 (mm)	6 (mm)	30 (mm)	12x9
Checkerboard Coarse	800x600 (mm)	6 (mm)	60 (mm)	12x9

Placement

The specific placement of the calibration target depends on each hardware configuration, but the goal to keep in mind is to achieve precise and stable corner detections. Hence, it is recommended to place the calibration target close to the camera, with its longer size perpendicular to the forward axis of the camera, as illustrated in the picture below:

Target Printing

The calibration target must be rigid without any noticeable bending when flat on the ground. We recommend using aluminum/LDPE composite for the required flatness and stiffness.

Note

We recommend using calib.io to obtain high quality calibration targets. The targets used in this tutorial were purchased from calib.io.

Ensure that the printed target is:

• Non-glossy to minimize light reflections and glare.

• Black and white in color.

• Squares in the checker pattern are indeed squares and not rectangles.

Validation tip: Instead of measuring the dimensions of single squares, it is more accurate to measure the side of multiple squares at once (all if possible) and divide the distance by the number of squares measured.

YAML Target Description

One important input of this tool is the a YAML file (target_file.yaml) detailing the measurements of the calibration target. The sample below refers to 12x9 checkerboard with 30 mm squares and 6 mm thickness.

target_type: 'checkerboard'
num_squares_height: 9            # Number of squares in Y dimension
num_squares_width: 12            # Number of squares in X dimension
longer_side_m: 0.36              # Length in meters of the longest side of the checker pattern
thickness_m: 0.006               # Thickness of the calibration board in meters
num_still_frames: 10             # Number of consecutive still frames to use for pose estimation
still_image_pixel_threshold: 5   # Maximum pixel detection difference to consider still frame

Note

It is important to create the YAML file with accurate measurements of the calibration target being used.

Note

Some calibration tools use the inner checkerboard corners instead of the number of squares. Make sure to use the number of squares in width and height, or the calibration target will not be detected correctly.

Data Recording

Follow the guidelines below when recording the rosbag for ground calibration:

• Make sure that the ground around the robot is smooth and flat.

• Make sure that the calibration space is brightly illuminated with diffuse light, to avoid glare on the calibration target.

• Target dimensions, placement, and design must follow the guidelines in the Calibration Target section above.

• There must be only one calibration target in the scene.

• The calibration target must be static during the recording.

• The length of the recording is recommended to be between 5 seconds and 10 seconds.

• When using the Nova sensor set, it is recommended to use the isaac_ros_nova_recorder. Image topics are recorded with H.264 compression by default.

• When using a different data recorder (e.g. with non-Nova sensors), it is recommended to record non-compressed image topics. This tool only supports H.264 decompression.

• When using non-Nova sensors, configure the camera resolution to its highest setting.

• When using cameras with infrared (IR) projector (including but not limited to RealSense cameras) it is recommended to disable the IR emitter and other sources of IR light during data recording.

Refer to isaac_ros_data_recorder for additional details on data recording.

Note

Calibration target detection with IR cameras is not as reliable as with color cameras.

When using camera sets with both IR and color imagers (including but not limited to RealSense cameras), it is recommended to use a color imager for ground calibration. If it is a must to use an IR imager, it is recommended to follow accurately the instructions in the Calibration Target and Data Recording sections, and try different locations and sizes for the calibration target.

Troubleshooting

Isaac ROS Troubleshooting

For solutions to problems with Isaac ROS, see troubleshooting.

Image topic does not exist in the rosbag

Symptom

Nothing seems to happen until the end of the rosbag and nothing is visualized in the /camera_ground_calibration/target_detection topic.

Solution

Make sure to choose an existent topic in the rosbag. Topics in the rosbag can be listed with the command:

ros2 bag info <rosbag>

Target Not Detected

Symptom

Warning message reading Target not detected in the terminal and overlaid on the images streamed on the /camera_ground_calibration/target_detection topic.

[plane_target_calibration.py-3] [WARN] [1724803927.046786211] [camera_ground_calibration.plane_target_calibration]: Target not detected

Solution

Follow the guidelines in the Calibration Target and Data Recording sections above, and make sure that all the following are true:

• The provided image topic matches the camera visualizing the calibration target.

• The calibration target is fully in sight in the rectified / undistorted image.

• The calibration target is brightly and homogeneously illuminated without any glare.

• The calibration target is not too far away from the camera.

• The squares in the calibration target are not too small. Note that using large calibration targets may lead to the squares farther away from the camera to become too small in the image.

• The camera resolution is configured at its highest setting.

• The YAML file correctly defines the calibration target used.

Target Moving or Uncertain

Symptom

Message in the terminal reading:

[plane_target_calibration.py-3] [INFO] [1724803649.611023580] [camera_ground_calibration.plane_target_calibration]: Target moving or uncertain. Please keep target still. If it is not moving, place the target closer to the camera or print a larger target.

Solution

Take a recording with one static calibration target, and make sure its dimensions, placement, and design follow the guidelines in the Calibration Target section above.

API

Usage

ros2 launch isaac_ros_ground_calibration camera_ground_calibration.launch.py rosbag:=<rosbag> image_topic:=<camera_topic_name> target_file:=<target_info.yaml>  /[output:=<output.txt> /]

ROS Launch Arguments

ROS Launch Argument	Default Value	Description
rosbag	None	Path to recording.
image_topic	None	Image topic to use for calibration. It supports raw images or compressed with H.264.
target_file	None	Path to file YAML defining the calibration target.
rectify_width	None	Image width. To be set only if image is not already rectified/undistorted.
rectify_height	None	Image height. To be set only if image is not already rectified/undistorted.
output	None	(Optional) path to the output file. The content of the file matches the text in the terminal.
camera_info_topic	Inferred from image_topic, with suffix /camera_info	(Optional) camera info topic corresponding to image_topic.
h264_compressed_topic_suffix	/image_compressed	(Optional) topic suffix identifying topic names assumed to be encoded with H.264.

ROS Topics Subscribed

ROS Topic	Interface	Description
image_topic	sensor_msgs/msg/CompressedImage or sensor_msgs/msg/Image	Camera stream.
camera_info_topic	sensor_msgs/msg/CameraInfo	Camera intrinsics.

ROS Topics Published

ROS Topic	Interface	Description
/camera_ground_calibration/target_detection	sensor_msgs/msg/Image	Camera stream with overlaid detections.