Configure Cameras for DisplayPort Insertion#

  1. Complete Isaac ROS RealSense Setup. Connect the camera using a USB 3 port and cable, and position it so that the plug, socket, and robot workspace are visible. Avoid reflective surfaces and verify the depth image before running perception.

    Note

    If you already added the realsense image layer to the workspace configuration file in Configure the Workspace Container and ran isaac-ros activate --build-local, skip the Docker configuration and activation steps on that page.

  2. Identify each camera’s serial number:

    rs-enumerate-devices -S
    
  3. Choose a setup name. Copy unspecified.yaml to <SETUP_NAME>.yaml in the same source directory and fill in the serial numbers.

  4. Follow Isaac for Manipulation Camera Calibration for each camera. Assign distinct frames, such as camera_1_link and camera_2_link.

  5. In static_transforms.launch.py, add an entry named <SETUP_NAME> to calibrations_dict. Copy the measured translation and quaternion from the calibration result. Set parent_frame to the calibration’s robot reference frame and child_frame to the matching camera frame. Repeat for each camera.

  6. Set setup: <SETUP_NAME> in the DisplayPort workflow YAML. Use depth_type: REALSENSE and enable_dnn_depth_in_realsense: 'false' so FoundationPose receives the camera’s depth stream. Rebuild the source package and source the workspace after editing the configuration:

    cd ${ISAAC_ROS_WS}
    ISAAC_ROS_SKIP_ASSET_INSTALL=1 MANIPULATOR_INSTALL_ASSETS=0 \
       colcon build --symlink-install --packages-select isaac_ros_manipulation_bringup
    source install/setup.bash
    

Continue with the DisplayPort workflow for object meshes, model paths, and the plug and socket pose configuration.