Decoding Jetson H.264 Images on Non-NVIDIA Powered Systems

https://media.githubusercontent.com/media/NVIDIA-ISAAC-ROS/.github/main/resources/isaac_ros_docs/concepts/compression/h264/image_view_h264_decoded.png/

Overview

Using NVIDIA-accelerated Isaac ROS compression to H.264 encode data for playback through Isaac ROS H.264 decoder on NVIDIA-powered systems is fast and efficient. However, you may need to decode recorded data on systems that are not NVIDIA-powered. Fortunately, the output of the isaac_ros_h264_encoder package can be reformatted to work with other available ROS 2 decoder packages.

This tutorial shows you how to replay a rosbag recorded on a Jetson using a third-party H.264 decoder image_transport plugin used in FogROS2.

The launch file replays a target rosbag recorded with isaac_ros_h264_encoder. It is recorded through a node that reformats the compressed messages for compatibility. Then the third-party decoder is used to display it in an image view window.

Tutorial Walkthrough

  1. Finish the setup in the quickstart.

    1. Set up your development environment by following the instructions in getting started.

    2. Clone isaac_ros_common under ${ISAAC_ROS_WS}/src.

      cd ${ISAAC_ROS_WS}/src && \
   git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common.git

    3. (Optional) Install dependencies for any sensors you want to use by following the sensor-specific guides.

      Warning

      We strongly recommend installing all sensor dependencies before starting any quickstarts. Some sensor dependencies require restarting the Isaac ROS Dev container during installation, which will interrupt the quickstart process.

  2. Clone the following third-party repository into your workspace:

    cd ${ISAAC_ROS_WS}/src && \
   git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_compression.git && \
   git clone https://github.com/clydemcqueen/h264_image_transport.git

  3. Launch the Docker container using the run_dev.sh script:

    cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
  ./scripts/run_dev.sh

  4. Install dependencies for the third-party package:

    sudo apt install libavdevice-dev libavformat-dev libavcodec-dev libavutil-dev libswscale-dev

  5. Build the third-party h264_image_transport package:

    cd ${ISAAC_ROS_WS} && \
  colcon build --symlink-install --packages-up-to \
    h264_image_transport isaac_ros_to_h264_msgs_packet && \
  source install/setup.bash

  6. Launch the graph to bring up an image viewer that shows the decoded output.

    ros2 launch isaac_ros_to_h264_msgs_packet h264_compatible_decoder.launch.py rosbag_path:=/workspaces/isaac_ros-dev/src/isaac_ros_compression/resources/rosbags/h264_compressed_sample.bag

    Note

    You may have to click on the “/image” to trigger window resizing.