isaac_ros_h264_encoder

Source code on GitHub.

Quickstart

Set Up Development Environment

  1. Set up your development environment by following the instructions in getting started.

  2. Clone isaac_ros_common under ${ISAAC_ROS_WS}/src.

    cd ${ISAAC_ROS_WS}/src && \
       git clone -b release-3.1 https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common.git isaac_ros_common
    
  3. (Optional) Install dependencies for any sensors you want to use by following the sensor-specific guides.

    Warning

    We strongly recommend installing all sensor dependencies before starting any quickstarts. Some sensor dependencies require restarting the Isaac ROS Dev container during installation, which will interrupt the quickstart process.

Download Quickstart Assets

  1. Download quickstart data from NGC:

    Make sure required libraries are installed.

    sudo apt-get install -y curl jq tar
    

    Then, run these commands to download the asset from NGC:

    NGC_ORG="nvidia"
    NGC_TEAM="isaac"
    PACKAGE_NAME="isaac_ros_h264_encoder"
    NGC_RESOURCE="isaac_ros_h264_encoder_assets"
    NGC_FILENAME="quickstart.tar.gz"
    MAJOR_VERSION=3
    MINOR_VERSION=1
    VERSION_REQ_URL="https://catalog.ngc.nvidia.com/api/resources/versions?orgName=$NGC_ORG&teamName=$NGC_TEAM&name=$NGC_RESOURCE&isPublic=true&pageNumber=0&pageSize=100&sortOrder=CREATED_DATE_DESC"
    AVAILABLE_VERSIONS=$(curl -s \
        -H "Accept: application/json" "$VERSION_REQ_URL")
    LATEST_VERSION_ID=$(echo $AVAILABLE_VERSIONS | jq -r "
        .recipeVersions[]
        | .versionId as \$v
        | \$v | select(test(\"^\\\\d+\\\\.\\\\d+\\\\.\\\\d+$\"))
        | split(\".\") | {major: .[0]|tonumber, minor: .[1]|tonumber, patch: .[2]|tonumber}
        | select(.major == $MAJOR_VERSION and .minor <= $MINOR_VERSION)
        | \$v
        " | sort -V | tail -n 1
    )
    if [ -z "$LATEST_VERSION_ID" ]; then
        echo "No corresponding version found for Isaac ROS $MAJOR_VERSION.$MINOR_VERSION"
        echo "Found versions:"
        echo $AVAILABLE_VERSIONS | jq -r '.recipeVersions[].versionId'
    else
        mkdir -p ${ISAAC_ROS_WS}/isaac_ros_assets && \
        FILE_REQ_URL="https://api.ngc.nvidia.com/v2/resources/$NGC_ORG/$NGC_TEAM/$NGC_RESOURCE/\
    versions/$LATEST_VERSION_ID/files/$NGC_FILENAME" && \
        curl -LO --request GET "${FILE_REQ_URL}" && \
        tar -xf ${NGC_FILENAME} -C ${ISAAC_ROS_WS}/isaac_ros_assets && \
        rm ${NGC_FILENAME}
    fi
    

Build isaac_ros_h264_encoder

  1. Launch the Docker container using the run_dev.sh script:

    cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
    ./scripts/run_dev.sh
    
  2. Install the prebuilt Debian package:

    sudo apt-get install -y ros-humble-isaac-ros-h264-encoder
    

Run Launch File

  1. Ensure that you have already set up your RealSense camera using the RealSense setup tutorial. If you have not, please set up the sensor and then restart this quickstart from the beginning.

  2. Continuing inside the container, install the following dependencies:

    sudo apt-get install -y ros-humble-isaac-ros-examples ros-humble-isaac-ros-realsense
    
  3. Run the launch file. This launch file launches the example with the RealSense camera:

    ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=realsense_stereo_rect,stereo_h264_encoder
    

Visualize Results

Note

Visualization is optional. To visualize the encoded output, you need to complete the Isaac ROS H264 decoder.

  1. Open a new terminal inside the Docker container:

    cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
       ./scripts/run_dev.sh
    
    1. Decode and visualize the images:

      ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=stereo_h264_decoder
      
      ros2 run image_view image_view --ros-args -r image:=/left/image_uncompressed
      
      ros2 run image_view image_view --ros-args -r image:=/right/image_uncompressed
      

      For example, the result looks like:

      https://media.githubusercontent.com/media/NVIDIA-ISAAC-ROS/.github/main/resources/isaac_ros_docs/concepts/compression/h264/realsense_example.png/

API

ROS Parameters

ROS Parameter

Type

Default

Description

input_width

uint32_t

1920

The width of the input image.

input_height

uint32_t

1200

The height of the input image.

qp

uint32_t

20

The encoder constant QP value.

hw_preset

uint32_t

0

The encoder hardware preset type. The value can be an integer from 0 to 3, representing Ultrafast, Fast, Medium and, Slow, respectively.

profile

uint32_t

0

The profile to be used for encoding. The value can be an integer from 0 to 2, representing Main, Baseline, and High, respectively.

iframe_interval

int32_t

5

Interval between two I frames, in number of frames. E.g., iframe_interval=5 represents IPPPPI..., iframe_interval=1 represents I frame only

config

std::string

pframe_cqp

A preset combination of qp, hw_preset, profile and iframe_interval. The value can be iframe_cqp, pframe_cqp or custom. When iframe_cqp or pframe_cqp is used, the default value of these four parameters will be applied. Only when this field is custom will the custom values will be used.

ROS Topics Subscribed

ROS Topic

Type

Description

image_raw

sensor_msgs/Image

Raw input image.

Warning

All input images are required to have height and width that are both an even number of pixels.

ROS Topics Published

ROS Topic

Interface

Description

image_compressed

sensor_msgs/CompressedImage

H.264 compressed image.

Input Restrictions

  1. The input image resolution must be the same as the dimension you provided, and the resolution must be no larger than ``1920x1200``.

  2. The input image should be in rgb8 or bgr8 format, and it will be converted to nv12 format before being sent to the encoder.

Output Interpretations

  1. The encoder could perform All-I frame or P-frame encoding and output the H.264 compressed data. The input and output are one-to-one mapped.