5. Sensors Architecture: General Requirements

5.1. Multi-camera Calibration Requirements

Req ID:

Coverage Overlap Requirements

27400300

For the purpose of extrinsic calibration between a pair of cameras in the camera belt, it is required to have sufficient coverage overlap between the two cameras. The overlap shall allow reliable detection of a checkerboard target with both cameras at the same target location, following Isaac extrinsic calibration procedure. The image below shows a successful detection of the target with one of the cameras. We have validated that a successful extrinsic calibration is achieved when the cameras have 25deg overlap of the covered field of view.

5.2. Nominal values and URDF file generation

Nominal values

In cases when the mechanical assembly of the camera(s) arrangement can meet angular tolerance requirement of ±1.6 degrees between any pair of camera modules, Isaac software modules, including Isaac Perceptor, can perform well provided a matching nominal calibration file in URDF format.

The advantage of this approach is that it does not require calibration between the camera modules and therefore reduces the time for Isaac software evaluation.

URDF file generation

Note

This section details the URDF creation process for Nova Orin Developer Kit, for illustration purposes only. It is not necessary to perform these steps if the sensors are mounted on the Nova Orin Developer Kit according to the table below. A default xacro file is available in nova_developer_kit_macro.urdf.xacro.

If the Nova Orin Developer Kit is mounted on a robot, it is still necessary to provide a URDF encoding its position on the robot. Follow the instructions Mounting the Nova Orin Developer Kit on a Robot.

The process of generation of ROS URDF nominal file follows the procedure below:

Vortex (or principal point) nominal location and orientation should be obtained from the vendor camera module documentation, for both stereo and mono camera modules.

Camera modules should be positioned in an optimal way on the target robot in the robot CAD model, following the guidelines and the requirements of this design guide.

Vortex location and orientation of each camera module with respect to the robot coordinate system should be summarized in a table of six degrees of freedom.

Using this table, URDF file should be generated using a standard process (the reader is referred to the available ROS URDF documentation, as well as to the xacro template examples for placing sensors on the Nova Orin Developer Kit nova_developer_kit_macro.urdf.xacro and for placing the Nova Orin Developer Kit on a robot nova_developer_kit.urdf.xacro).

The obtained URDF file should be referenced by Isaac software modules following Isaac Perceptor documentation (Mounting the Nova Orin Developer Kit on a Robot).

Vertexes table example

In this example we consider a rig that includes 3 Hawk camera and 3 Owl cameras, as depicted in the image below:

The thick green arrow indicates the location of the rig vertex, aligned with the center of the front Hawk camera, along the intersection line between the bottom and the front Hawk planes.

The other six small Cartesian axes indicate the locations of the vertexes of the six camera modules.

The figure also defines the rig coordinate system.

In this case, the vertex nominals table in rig coordinate system is provided below:

Camera	X [mm]	Y [mm]	Z [mm]	Yaw [deg]	Pitch [deg]	Roll [deg]
Hawk - Front	-16	75	12.5	0	0	0
Hawk - Left	-165	106	12.5	90	0	0
Hawk - Right	-15	-106	12.5	-90	0	0
Owl - Front	-10	0	38	0	0	0
Owl - Left	-90	112	38	90	0	0
Owl - Right	-90	-112	38	-90	0	0

The corresponding xacro file for Nova Orin Developer Kit, as well as Hawk, and Owl cameras are available in nova_developer_kit_macro.urdf.xacro, hawk_macro.urdf.xacro, and owl_macro.urdf.xacro, respectively. The reader is advised to download Hawk and Owl specifications and 3D models to cross-check these numbers.

URDF Mounting the Nova Orin Developer Kit on a Robot

If the Nova Orin Developer Kit is mounted on a robot, the user must provide a URDF encoding its position on the robot.

It is recommended to follow the self-contained calibration instructions. The output of calibration is stored in a default location in the robot, and Isaac Perceptor will use it without requiring any additional arguments.

Alternatively, the remaining of this section details how to create a robot nominals URDF file (urdf_nominals_file_path) to be used with Isaac Perceptor. Note that the steps below shall not be followed if performing calibration. The instructions Isaac Perceptor on a Robot with the Nova Orin Developer Kit detail how to provide the urdf_nominals_file_path to Isaac Perceptor.

To define your own URDF nominals file to use in Isaac Perceptor, you can use the following code snippet and store it at a suitable urdf_nominals_file_path:

<?xml version="1.0"?>
<robot name="please_specify_robot_name" xmlns:xacro="http://www.ros.org/wiki/xacro">
<xacro:arg name="robot_namespace" default="/"/>
<xacro:include filename="$(find nova_developer_kit_description)/urdf/nova_developer_kit_macro.urdf.xacro"/>

<link name="base_link"/>
<xacro:nova_developer_kit name="nova_developer_kit" parent="base_link" mount_is_root="true">
    <origin xyz="0.0 0.0 0.0" rpy="0.0 0.0 0.0"/>
</xacro:nova_developer_kit>
</robot>

<!-- Your Robot Description goes here. -->

The fields xyz and rpy parametrize the transformation between the Nova Orin Developer Kit and the provided parent link (e.g. base_link). Specifically, the field xyz represents a 3D translation in meters, and the field rpy represents a rotation as fixed-axis Euler angles in radians: roll, pitch and yaw, respectively.

The field mount_is_root must be set to true if the transformation is provided with respect to the Nova Orin Developer Kit mount point (center screw of the lower plate), or false if the transformation refers to the Nova Orin Developer Kit main link (center-front of the module).

For example, if you are mounting the Nova Orin Developer Kit such that its center-front point is at a height of Z = 0.4 m above the robot origin and X = -0.1 m behind it, the root link origin should be adjusted accordingly:

...
<xacro:nova_developer_kit name="nova_developer_kit" parent="base_link" mount_is_root="false">
    <origin xyz="-0.1 0.0 0.4" rpy="0.0 0.0 0.0"/>
...