`isaac_ros_h264_decoder`#

Source code available on GitHub.

Quickstart#

This quickstart shows an example of how to use the isaac_ros_h264_decoder with a pre-recorded rosbag, which contains compressed H.264 images generated from isaac_ros_h264_encoder with two Argus cameras as the input source. You will be able to visualize the decoded images after the last step.

Set Up Development Environment#

Set up your development environment by following the instructions in getting started.
(Optional) Install dependencies for any sensors you want to use by following the sensor-specific guides.

Note

We strongly recommend installing all sensor dependencies before starting any quickstarts. Some sensor dependencies require restarting the development environment during installation, which will interrupt the quickstart process.

Download Quickstart Assets#

Download quickstart data from NGC:

NVIDIA Internal:

The NGC CLI must be set up once per system.
Install the NGC CLI from here.
Note

Jetson Only: Do not edit the ~/.bash_profile file as suggested by the official NGC CLI instructions.

Instead, add the NGC CLI to your PATH through ~/.bashrc:
s="export PATH=\"\$PATH:$(pwd)/ngc-cli\""
f="$HOME/.bashrc"; grep -qxF "$s" $f || echo "$s" | tee -a $f && source $f
Generate an API key here.
Run ngc config set:

When prompted, enter the API key you generated earlier.
ngc config set
To confirm that the NGC CLI is set up correctly, run the following command:
grep -E "^apikey\s*=\s*" ~/.ngc/config
You should see an output with a valid API key:
apikey = <your_api_key>

Make sure required libraries are installed.

sudo apt-get install -y curl jq tar

Then, run these commands to download the asset from NGC:

NGC_ORG="nvstaging"
NGC_TEAM="isaac"
PACKAGE_NAME="isaac_ros_h264_decoder"
NGC_RESOURCE="isaac_ros_h264_decoder_assets"
NGC_FILENAME="quickstart.tar.gz"
MAJOR_VERSION=3
MINOR_VERSION=3
NGC_CONFIG_FILE="$HOME/.ngc/config"
API_KEY=$(grep -E "^apikey\s*=\s*" $NGC_CONFIG_FILE | \
    awk -F "=" '{print $2}' | tr -d '[:space:]')
TOKEN=$(curl -s -u "\$oauthtoken":"$API_KEY" -H 'Accept:application/json' \
    "https://authn.nvidia.com/token?service=ngc"\
"&scope=group/ngc:$NGC_ORG&group/ngc:$NGC_ORG/$NGC_TEAM" | \
    jq -r '.token')
VERSION_REQ_URL="https://api.ngc.nvidia.com/v2/org/$NGC_ORG/team/$NGC_TEAM/resources/$NGC_RESOURCE/versions"
AVAILABLE_VERSIONS=$(curl -s -H "Authorization: Bearer ${TOKEN}" \
    -H "Accept: application/json" "$VERSION_REQ_URL")
LATEST_VERSION_ID=$(echo $AVAILABLE_VERSIONS | jq -r "
    .recipeVersions[]
    | .versionId as \$v
    | \$v | select(test(\"^\\\\d+\\\\.\\\\d+\\\\.\\\\d+$\"))
    | split(\".\") | {major: .[0]|tonumber, minor: .[1]|tonumber, patch: .[2]|tonumber}
    | select(.major == $MAJOR_VERSION and .minor <= $MINOR_VERSION)
    | \$v
    " | sort -V | tail -n 1
)
if [ -z "$LATEST_VERSION_ID" ]; then
    echo "No corresponding version found for Isaac ROS $MAJOR_VERSION.$MINOR_VERSION"
    echo "Found versions:"
    echo $AVAILABLE_VERSIONS | jq -r '.recipeVersions[].versionId'
else
    mkdir -p ${ISAAC_ROS_WS}/isaac_ros_assets && \
    FILE_REQ_URL="https://api.ngc.nvidia.com/v2/org/$NGC_ORG/team/$NGC_TEAM/resources/\
$NGC_RESOURCE/versions/$LATEST_VERSION_ID/files/$NGC_FILENAME" && \
    curl -LO --request GET "${FILE_REQ_URL}" \
        -H "Authorization: Bearer ${TOKEN}" \
        -H "Content-Type: application/json" && \
    tar -xf ${NGC_FILENAME} -C ${ISAAC_ROS_WS}/isaac_ros_assets && \
    rm ${NGC_FILENAME}
fi

Build `isaac_ros_h264_decoder`#

Launch the Docker container using the run_dev.py script:

cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
./scripts/run_dev.py

Install the prebuilt Debian package:

NVIDIA Internal: Run these commands to add the internal apt repository:

sudo apt install curl -y
k="/usr/share/keyrings/nvidia-isaac-ros.gpg"
curl -fsSL https://isaac.download.nvidia.com/isaac-ros/repos.key | sudo gpg --dearmor | sudo tee -a $k > /dev/null
f="/etc/apt/sources.list.d/nvidia-isaac-ros.list" && sudo touch $f
s="deb [signed-by=$k] https://urm.nvidia.com/artifactory/sw-isaac-staging-debian-local jammy release-3.3"
grep -qxF "$s" $f || echo "$s" | sudo tee -a $f

pin_content=$'package: *\nPin: origin isaac.download.nvidia.com\nPin-Priority: 400'
echo "$pin_content" | sudo tee /etc/apt/preferences.d/isaac-ros

sudo apt-get update

sudo apt-get install -y ros-humble-isaac-ros-h264-decoder

Clone this repository under ${ISAAC_ROS_WS}/src:

cd ${ISAAC_ROS_WS}/src && \
   git clone -b release-3.3 git@github.com:NVIDIA-ISAAC-ROS/validation-isaac_ros_compression.git isaac_ros_compression

Launch the Docker container using the run_dev.py script:

cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
./scripts/run_dev.py

Use rosdep to install the package’s dependencies:

NVIDIA Internal: Run these commands to add the internal apt repository:

sudo apt install curl -y
k="/usr/share/keyrings/nvidia-isaac-ros.gpg"
curl -fsSL https://isaac.download.nvidia.com/isaac-ros/repos.key | sudo gpg --dearmor | sudo tee -a $k > /dev/null
f="/etc/apt/sources.list.d/nvidia-isaac-ros.list" && sudo touch $f
s="deb [signed-by=$k] https://urm.nvidia.com/artifactory/sw-isaac-staging-debian-local jammy release-3.3"
grep -qxF "$s" $f || echo "$s" | sudo tee -a $f

pin_content=$'package: *\nPin: origin isaac.download.nvidia.com\nPin-Priority: 400'
echo "$pin_content" | sudo tee /etc/apt/preferences.d/isaac-ros

sudo apt-get update

rosdep update && rosdep install --from-paths ${ISAAC_ROS_WS}/src/isaac_ros_compression/isaac_ros_h264_decoder --ignore-src -y

Build the package from source:

cd ${ISAAC_ROS_WS}/ && \
   colcon build --symlink-install --packages-up-to isaac_ros_h264_decoder --base-paths ${ISAAC_ROS_WS}/src/isaac_ros_compression/isaac_ros_h264_decoder

Source the ROS workspace:

Note

Make sure to repeat this step in every terminal created inside the Docker container.

Because this package was built from source, the enclosing workspace must be sourced for ROS to be able to find the package’s contents.
```
source install/setup.bash
```

Run Launch File#

Continuing inside the Docker container, install the following dependencies:

NVIDIA Internal: Run these commands to add the internal apt repository:

sudo apt install curl -y
k="/usr/share/keyrings/nvidia-isaac-ros.gpg"
curl -fsSL https://isaac.download.nvidia.com/isaac-ros/repos.key | sudo gpg --dearmor | sudo tee -a $k > /dev/null
f="/etc/apt/sources.list.d/nvidia-isaac-ros.list" && sudo touch $f
s="deb [signed-by=$k] https://urm.nvidia.com/artifactory/sw-isaac-staging-debian-local jammy release-3.3"
grep -qxF "$s" $f || echo "$s" | sudo tee -a $f

pin_content=$'package: *\nPin: origin isaac.download.nvidia.com\nPin-Priority: 400'
echo "$pin_content" | sudo tee /etc/apt/preferences.d/isaac-ros

sudo apt-get update

sudo apt-get install -y ros-humble-isaac-ros-examples

Run the following launch file to spin up a demo of this package using the quickstart rosbag:

ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=stereo_h264_decoder

Open a second terminal inside the Docker container:

cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
./scripts/run_dev.py

Run the rosbag file to simulate an image stream:

ros2 bag play -l ${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_h264_decoder/quickstart.bag

Visualize Results#

Open a new terminal inside the Docker container:

cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
   ./scripts/run_dev.py

Visualize and validate the output of the package:

ros2 run image_view image_view --ros-args -r image:=/left/image_uncompressed

ros2 run image_view image_view --ros-args -r image:=/right/image_uncompressed

Troubleshooting#

Symptom#

Launching the decoder node using ros2 launch isaac_ros_h264_decoder isaac_ros_h264_decoder_rosbag.launch.py, produces the below error message:

x86_64 Platform:

[component_container_mt-2] 2023-04-04 22:19:02.609 WARN  /workspaces/isaac_ros-dev/ros_ws/src/isaac_ros_compression/isaac_ros_h264_decoder/gxf/codec/components/decoder.cpp@310: Decoder output is empty, current frame will be dropped

Jetson Platform:

[component_container_mt-2] 2023-04-04 06:39:57.766 WARN  gxf/std/greedy_scheduler.cpp@235: Error while executing entity 20 named 'KIWPUYFVJT_decoder_response': GXF_FAILURE
[component_container_mt-2] [ERROR] (/usr/src/jetson_multimedia_api/samples/common/classes/NvV4l2ElementPlane.cpp:178) <dec0> Capture Plane:Error while DQing buffer: Invalid argument
[component_container_mt-2] 2023-04-04 06:39:57.766 ERROR /workspaces/isaac_ros-dev/ros_ws/src/isaac_ros_compression/isaac_ros_h264_decoder/gxf/codec/components/decoder_request.cpp@530: Failed to dequeue buffer from capture plane
[component_container_mt-2] 2023-04-04 06:39:57.766 ERROR gxf/std/entity_executor.cpp@509: Failed to tick codelet  in entity: KIWPUYFVJT_decoder_response code: GXF_FAILURE
[component_container_mt-2] 2023-04-04 06:39:57.766 ERROR gxf/std/entity_executor.cpp@203: Entity with eid 35 not found!

Solution#

You may see these errors when decoder receives P-frames first. Decoder can not decode P-frames until it has received the first I-frame. You can start the decoder before playing the rosbag to avoid these errors.

API#

Usage#

ros2 launch isaac_ros_h264_decoder isaac_ros_h264_decoder.launch.py

ROS Parameters#

ROS Topics Subscribed#

ROS Topic	Interface	Description
`image_compressed`	sensor_msgs/CompressedImage	The H.264 compressed image

ROS Topics Published#

ROS Topic	Interface	Description
`image_uncompressed`	sensor_msgs/Image	The uncompressed image with RGB8 encoding.

Output Interpretations#

The isaac_ros_h264_decoder package outputs an uncompressed image with the same resolution as the original image. The output image will be in RGB8 format.

isaac_ros_h264_decoder#

Quickstart#

Set Up Development Environment#

Download Quickstart Assets#

Build isaac_ros_h264_decoder#

Run Launch File#

Visualize Results#

Troubleshooting#

Symptom#

Solution#

API#

Usage#

ROS Parameters#

ROS Topics Subscribed#

ROS Topics Published#

Output Interpretations#

`isaac_ros_h264_decoder`#

Build `isaac_ros_h264_decoder`#