============== |package_name| ============== :ir_github:` ` Overview -------- The |package_name| package enables calibration of any color or grayscale camera with respect to the ground plane. Autonomous robotic systems need to perceive obstacles and the environment around them. Ground calibration helps perception modules to identify the ground plane, and differentiate it from objects and other features in the scene. Ground calibration refers to the 3 Degrees of Freedom (DoF) aligning a sensor or a robot with the ground plane, and includes elevation in the Z axis, as well as pitch and roll rotations. This package is expected to be used offline and requires a calibration target on the ground. Refer to the :ref:`Calibration Target ` section below for printing instructions. The main inputs of the camera-ground calibration tool are a recording of a scene including **one static** calibration target placed on **flat ground** (refer to the :ref:`Data Recording ` section below), the measurements of the calibration target in a :ref:`YAML file `, and the topic of the camera observing the target. The output of this tool is a 3-DoF transformation between a camera in canonical frame (`X forward, Y left, Z up `_) and the ground plane. A sample output is shown below: .. code:: xml Plane calibration results (3-DoF): plane_T_front_stereo_camera_left: z: 0.339673 m, roll: -0.006314 rad, pitch: -0.020137 rad Calibration can be substituted as follows in a URDF file (using nominal values for the remaining non-estimated 3 DoF): Quickstart ---------- Set Up Development Environment ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. include:: /_snippets/set_up_dev_env.rst Download Quickstart Assets ^^^^^^^^^^^^^^^^^^^^^^^^^^ 1. Download quickstart data from NGC: :ir_assets:` ` Build |package_name| ^^^^^^^^^^^^^^^^^^^^ .. tabs:: .. tab:: Binary Package #. Launch the Docker container using the ``run_dev.sh`` script: .. code:: bash cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \ ./scripts/run_dev.sh #. Install the prebuilt Debian package: :ir_apt: .. code:: bash sudo apt update sudo apt-get install -y ros-humble-isaac-ros-ground-calibration #. Declare ``ROS_DOMAIN_ID`` with the same unique ID (`number between 0 and 101 `__) on every bash instance inside the Docker container: .. code:: bash export ROS_DOMAIN_ID= .. tab:: Build from Source #. Clone this repository under ``${ISAAC_ROS_WS}/src``: .. code:: bash cd ${ISAAC_ROS_WS}/src && \ git clone :ir_clone:`` #. Launch the Docker container using the ``run_dev.sh`` script: .. code:: bash cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \ ./scripts/run_dev.sh #. Use ``rosdep`` to install the package's dependencies: :ir_apt: .. code:: bash rosdep update && rosdep install -i -r --from-paths ${ISAAC_ROS_WS}/src/isaac_ros_nova/isaac_ros_ground_calibration/ --rosdistro humble -y #. Build the package from source: .. code:: bash cd ${ISAAC_ROS_WS} && \ colcon build --symlink-install --packages-up-to isaac_ros_ground_calibration #. Source the ROS workspace: .. note:: Make sure to repeat this step in **every** terminal created inside the Docker container. Because this package was built from source, the enclosing workspace must be sourced for ROS to be able to find the package's contents. .. code:: bash cd ${ISAAC_ROS_WS} && \ source install/setup.bash #. Declare ``ROS_DOMAIN_ID`` with the same unique ID (`number between 0 and 101 `__) on every bash instance inside the Docker container: .. code:: bash export ROS_DOMAIN_ID= Run Launch File ^^^^^^^^^^^^^^^ 1. Continuing inside the Docker container, run the following launch file to spin up a demo of this package: .. code:: bash ros2 launch isaac_ros_ground_calibration camera_ground_calibration.launch.py \ rosbag:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_ground_calibration/quickstart_rosbag \ image_topic:=/front_stereo_camera/left/image_compressed \ target_file:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_ground_calibration/quickstart_target_info.yaml \ rectify_width:=1920 rectify_height:=1200 (Optional) Add an output file path to the launch command to keep a copy of the calibration results, using the ``output`` argument. .. note:: The arguments ``rectify_width`` and ``rectify_height`` must reflect the image resolution of the camera used, and they **only** need to be provided if the image stream is not already rectified. Visualize ^^^^^^^^^ #. Open a **new** terminal inside the Docker container: .. code:: bash cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \ ./scripts/run_dev.sh #. Declare ``ROS_DOMAIN_ID`` with the **same** unique ID as in the main terminal: .. code:: bash export ROS_DOMAIN_ID= #. Run RViz or ``rqt_image_view`` to visualize the detection and pose estimation of the calibration target. By default, detections will be published in ``/camera_ground_calibration/target_detection``. **Images will stop being published after ground calibration succeeds**: .. code:: bash ros2 run rqt_image_view rqt_image_view .. figure:: :ir_lfs:`` :width: 800px :alt: ground_calibration_target_detection :align: center Calibration Target ------------------ Supported Patterns ^^^^^^^^^^^^^^^^^^ The current version of the camera-ground calibration tool only supports checkerboards. .. note:: Checkerboards must be fully on sight of the camera for them to be detected. Partial detections are not supported. Size and Design ^^^^^^^^^^^^^^^ The required size of the calibration target depends on the camera configuration and sensor placement in the robot, including but not limited to distance to the ground, incidence angle, image resolution, and lens choice. As a general guideline it is recommended to use a calibration target with large square size, and a minimum of 7 rows and 11 columns. Large calibration targets are preferred, as long as the squares that are further away from the camera still appear large in the image and their detection is stable. Hence, the larger the target, the larger the square size needs to be. Moreover, the smaller the image resolution used for target detection, the larger the square size needs to be. The specifications for the calibration targets used in this tutorial are as follows: =================== =============== ========= =========== ======================= Pattern Type Board Dimension Thickness Square Size Checkerboard Resolution =================== =============== ========= =========== ======================= Checkerboard Coarse 400x300 (mm) 6 (mm) 30 (mm) 12x9 Checkerboard Coarse 800x600 (mm) 6 (mm) 60 (mm) 12x9 =================== =============== ========= =========== ======================= Placement ^^^^^^^^^ The specific placement of the calibration target depends on each hardware configuration, but the goal to keep in mind is to achieve precise and stable corner detections. Hence, it is recommended to place the calibration target close to the camera, with its longer size perpendicular to the forward axis of the camera, as illustrated in the picture below: .. figure:: :ir_lfs:`` :width: 800px :alt: ground_calibration_board_placement :align: center Target Printing ^^^^^^^^^^^^^^^ The calibration target must be rigid without any noticeable bending when flat on the ground. We recommend using aluminum/LDPE composite for the required flatness and stiffness. .. note:: We recommend using `calib.io `__ to obtain high quality calibration targets. The targets used in this tutorial were purchased from calib.io. Ensure that the printed target is: * Non-glossy to minimize light reflections and glare. * Black and white in color. * Squares in the checker pattern are indeed squares and not rectangles. **Validation tip**: Instead of measuring the dimensions of single squares, it is more accurate to measure the side of multiple squares at once (all if possible) and divide the distance by the number of squares measured. .. _YAML Target Description: YAML Target Description ^^^^^^^^^^^^^^^^^^^^^^^ One important input of this tool is the a YAML file (``target_file.yaml``) detailing the measurements of the calibration target. The sample below refers to 12x9 checkerboard with 30 mm squares and 6 mm thickness. .. code:: yaml target_type: 'checkerboard' num_squares_height: 9 # Number of squares in Y dimension num_squares_width: 12 # Number of squares in X dimension longer_side_m: 0.36 # Length in meters of the longest side of the checker pattern thickness_m: 0.006 # Thickness of the calibration board in meters num_still_frames: 10 # Number of consecutive still frames to use for pose estimation still_image_pixel_threshold: 5 # Maximum pixel detection difference to consider still frame .. note:: It is important to create the YAML file with accurate measurements of the calibration target being used. .. note:: Some calibration tools use the inner checkerboard corners instead of the number of squares. Make sure to use the number of squares in width and height, or the calibration target will not be detected correctly. .. _Data Recording for Ground Calibration: Data Recording -------------- Follow the guidelines below when recording the rosbag for ground calibration: * Make sure that the ground around the robot is smooth and flat. * Make sure that the calibration space is brightly illuminated with diffuse light, to avoid glare on the calibration target. * Target dimensions, placement, and design must follow the guidelines in the :ref:`Calibration Target ` section above. * There must be only **one** calibration target in the scene. * The calibration target must be static during the recording. * The length of the recording is recommended to be between 5 seconds and 10 seconds. * When using the Nova sensor set, it is recommended to use the :doc:`isaac_ros_nova_recorder `. Image topics are recorded with H.264 compression by default. * When using a different data recorder (e.g. with non-Nova sensors), it is recommended to record **non-compressed** image topics. This tool only supports H.264 decompression. * When using non-Nova sensors, configure the camera resolution to its highest setting. * When using cameras with infrared (IR) projector (including but not limited to RealSense cameras) it is recommended to disable the IR emitter and other sources of IR light during data recording. Refer to :doc:`isaac_ros_data_recorder ` for additional details on data recording. .. note:: Calibration target detection with IR cameras is not as reliable as with color cameras. When using camera sets with both IR and color imagers (including but not limited to RealSense cameras), it is recommended to use a color imager for ground calibration. If it is a must to use an IR imager, it is recommended to follow accurately the instructions in the :ref:`Calibration Target ` and :ref:`Data Recording ` sections, and try different locations and sizes for the calibration target. Troubleshooting --------------- Isaac ROS Troubleshooting ^^^^^^^^^^^^^^^^^^^^^^^^^ For solutions to problems with Isaac ROS, see :doc:`troubleshooting `. Image topic does not exist in the rosbag ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Symptom ~~~~~~~ Nothing seems to happen until the end of the rosbag and nothing is visualized in the ``/camera_ground_calibration/target_detection`` topic. Solution ~~~~~~~~ Make sure to choose an existent topic in the rosbag. Topics in the rosbag can be listed with the command: .. code:: bash ros2 bag info Target Not Detected ^^^^^^^^^^^^^^^^^^^ Symptom ~~~~~~~ Warning message reading ``Target not detected`` in the terminal and overlaid on the images streamed on the ``/camera_ground_calibration/target_detection`` topic. .. code:: bash [plane_target_calibration.py-3] [WARN] [1724803927.046786211] [camera_ground_calibration.plane_target_calibration]: Target not detected Solution ~~~~~~~~ Follow the guidelines in the :ref:`Calibration Target ` and :ref:`Data Recording ` sections above, and make sure that all the following are true: * The provided image topic matches the camera visualizing the calibration target. * The calibration target is fully in sight in the rectified / undistorted image. * The calibration target is brightly and homogeneously illuminated without any glare. * The calibration target is not too far away from the camera. * The squares in the calibration target are not too small. Note that using large calibration targets may lead to the squares farther away from the camera to become too small in the image. * The camera resolution is configured at its highest setting. * The :ref:`YAML file ` correctly defines the calibration target used. Target Moving or Uncertain ^^^^^^^^^^^^^^^^^^^^^^^^^^ Symptom ~~~~~~~ Message in the terminal reading: .. code:: bash [plane_target_calibration.py-3] [INFO] [1724803649.611023580] [camera_ground_calibration.plane_target_calibration]: Target moving or uncertain. Please keep target still. If it is not moving, place the target closer to the camera or print a larger target. Solution ~~~~~~~~ Take a recording with **one** static calibration target, and make sure its dimensions, placement, and design follow the guidelines in the :ref:`Calibration Target ` section above. API ---- Usage ^^^^^ .. code:: bash ros2 launch isaac_ros_ground_calibration camera_ground_calibration.launch.py rosbag:= image_topic:= target_file:= /[output:= /] ROS Launch Arguments ^^^^^^^^^^^^^^^^^^^^ ============================================ ========================================================================================= ================================================================================================ ROS Launch Argument Default Value Description ============================================ ========================================================================================= ================================================================================================ ``rosbag`` None Path to recording. ``image_topic`` None Image topic to use for calibration. It supports raw images or compressed with H.264. ``target_file`` None Path to file YAML defining the calibration target. ``rectify_width`` None Image width. To be set **only** if image is not already rectified/undistorted. ``rectify_height`` None Image height. To be set **only** if image is not already rectified/undistorted. ``output`` None (Optional) path to the output file. The content of the file matches the text in the terminal. ``camera_info_topic`` Inferred from ``image_topic``, with suffix ``/camera_info`` (Optional) camera info topic corresponding to ``image_topic``. ``h264_compressed_topic_suffix`` ``/image_compressed`` (Optional) topic suffix identifying topic names assumed to be encoded with H.264. ============================================ ========================================================================================= ================================================================================================ ROS Topics Subscribed ^^^^^^^^^^^^^^^^^^^^^ ====================================== ================================================================================================================================================================================================================================================ ========================== ROS Topic Interface Description ====================================== ================================================================================================================================================================================================================================================ ========================== ``image_topic`` `sensor_msgs/msg/CompressedImage `__ or `sensor_msgs/msg/Image `__ Camera stream. ``camera_info_topic`` `sensor_msgs/msg/CameraInfo `__ Camera intrinsics. ====================================== ================================================================================================================================================================================================================================================ ========================== ROS Topics Published ^^^^^^^^^^^^^^^^^^^^ ===================================================== =============================================================================================================================================================== ================================================== ROS Topic Interface Description ===================================================== =============================================================================================================================================================== ================================================== ``/camera_ground_calibration/target_detection`` `sensor_msgs/msg/Image `__ Camera stream with overlaid detections. ===================================================== =============================================================================================================================================================== ================================================== .. |package_name| replace:: ``isaac_ros_ground_calibration``