5. Sensors Architecture: General Requirements

5.1. Multi-camera Calibration Requirements

Req ID:

Coverage Overlap Requirements

27400300

For the purpose of extrinsic calibration between a pair of cameras in the camera belt, it is required to have sufficient coverage overlap between the two cameras. The overlap shall allow reliable detection of a checkerboard target with both cameras at the same target location, following Isaac extrinsic calibration procedure. The image below shows a successful detection of the target with one of the cameras. We have validated that a successful extrinsic calibration is achieved when the cameras have 25deg overlap of the covered field of view.

5.2. Nominal values and URDF file generation

Nominal values

In cases when the mechanical assembly of the camera(s) arrangement can meet angular tolerance requirement of ±1.6 degrees between any pair of camera modules, Isaac software modules, including Isaac Perceptor, can perform well provided a matching nominal calibration file in URDF format.

The advantage of this approach is that it does not require calibration between the camera modules and therefore reduces the time for Isaac software evaluation.

URDF file generation

Note

This section details the URDF creation process for Nova Orin Developer Kit, for illustration purposes only. It is not necessary to perform these steps if the sensors are mounted on the Nova Orin Developer Kit according to the table below. A default xacro file is available in nova_developer_kit_macro.urdf.xacro.

If the Nova Orin Developer Kit is mounted on a robot, it is still necessary to provide a URDF encoding its position on the robot. Follow the instructions Mounting the Nova Orin Developer Kit on a Robot.

The process of generation of ROS URDF nominal file follows the procedure below:

Vortex (or principal point) nominal location and orientation should be obtained from the vendor camera module documentation, for both stereo and mono camera modules.

Camera modules should be positioned in an optimal way on the target robot in the robot CAD model, following the guidelines and the requirements of this design guide.

Vortex location and orientation of each camera module with respect to the robot coordinate system should be summarized in a table of six degrees of freedom.

Using this table, URDF file should be generated using a standard process (the reader is referred to the available ROS URDF documentation, as well as to the xacro template examples for placing sensors on the Nova Orin Developer Kit nova_developer_kit_macro.urdf.xacro and for placing the Nova Orin Developer Kit on a robot nova_developer_kit.urdf.xacro).

The obtained URDF file should be referenced by Isaac software modules following Isaac Perceptor documentation (Mounting the Nova Orin Developer Kit on a Robot).

Vertexes table example

In this example we consider a rig that includes 3 Hawk camera and 3 Owl cameras, as depicted in the image below:

The thick green arrow indicates the location of the rig vertex, aligned with the center of the front Hawk camera, along the intersection line between the bottom and the front Hawk planes.

The other six small Cartesian axes indicate the locations of the vertexes of the six camera modules.

The figure also defines the rig coordinate system.

In this case, the vertex nominals table in rig coordinate system is provided below:

Camera	X [mm]	Y [mm]	Z [mm]	Yaw [deg]	Pitch [deg]	Roll [deg]
Hawk - Front	-16	75	12.5	0	0	0
Hawk - Left	-165	106	12.5	90	0	0
Hawk - Right	-15	-106	12.5	-90	0	0
Owl - Front	-10	0	38	0	0	0
Owl - Left	-90	112	38	90	0	0
Owl - Right	-90	-112	38	-90	0	0

The corresponding xacro file for Nova Orin Developer Kit, as well as Hawk, and Owl cameras are available in nova_developer_kit_macro.urdf.xacro, hawk_macro.urdf.xacro, and owl_macro.urdf.xacro, respectively. The reader is advised to download Hawk and Owl specifications and 3D models to cross-check these numbers.