============== |package_name| ============== :ir_github:` ` .. figure:: :ir_lfs:`` :align: center Quickstart ---------- Set Up Development Environment ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. include:: /_snippets/set_up_dev_env.rst Datasets ^^^^^^^^ Download Quickstart Assets ~~~~~~~~~~~~~~~~~~~~~~~~~~ 1. Download quickstart data from NGC: :ir_assets:` ` Build |package_name| ^^^^^^^^^^^^^^^^^^^^ .. tabs:: .. tab:: Binary Package 1. Launch the Docker container using the ``run_dev.sh`` script: .. code:: bash cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \ ./scripts/run_dev.sh 2. Install the prebuilt Debian package: :ir_apt: .. code:: bash sudo apt-get install -y ros-humble-isaac-ros-image-proc .. tab:: Build from Source 1. Clone this repository under ``${ISAAC_ROS_WS}/src``: .. code:: bash cd ${ISAAC_ROS_WS}/src && \ git clone :ir_clone:`` 2. Launch the Docker container using the ``run_dev.sh`` script: .. code:: bash cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \ ./scripts/run_dev.sh 3. Use ``rosdep`` to install the package's dependencies: :ir_apt: .. code:: bash rosdep update && rosdep install --from-paths ${ISAAC_ROS_WS}/src/isaac_ros_image_pipeline/isaac_ros_image_proc --ignore-src -y 4. Build the package from source: .. code:: bash cd ${ISAAC_ROS_WS}/ && \ colcon build --symlink-install --packages-up-to isaac_ros_image_proc --base-paths ${ISAAC_ROS_WS}/src/isaac_ros_image_pipeline/isaac_ros_image_proc 5. Source the ROS workspace: .. note:: Make sure to repeat this step in **every** terminal created inside the Docker container. Because this package was built from source, the enclosing workspace must be sourced for ROS to be able to find the package's contents. .. code:: bash source install/setup.bash Run Launch File ^^^^^^^^^^^^^^^ .. note:: The quickstart for the alpha blending node has been integrated into ``isaac_ros_unet`` image segmentation. To run and visualize the results of alpha blending, see :doc:`here `. .. tabs:: .. tab:: Rosbag #. Continuing inside the Docker container, install the following dependencies: :ir_apt: .. code:: bash sudo apt-get install -y ros-humble-isaac-ros-examples 2. Run the following launch file to spin up a demo of this package using the quickstart rosbag: Resize: .. code:: bash ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=resize Color Conversion: .. code:: bash ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=color_conversion interface_specs_file:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_image_proc/quickstart_interface_specs.json Crop: .. code:: bash ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=crop interface_specs_file:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_image_proc/quickstart_interface_specs.json Rectify: .. code:: bash ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=rectify_mono interface_specs_file:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_image_proc/quickstart_interface_specs.json Flip: .. code:: bash ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=flip 2. Open a **second** terminal inside the Docker container: .. code:: bash cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \ ./scripts/run_dev.sh 3. Run the rosbag file to simulate an image stream: .. code:: bash ros2 bag play --loop ${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_image_proc/quickstart --remap /hawk_0_left_rgb_image:=/image_raw /hawk_0_left_rgb_camera_info:=/camera_info .. tab:: RealSense Camera #. Ensure that you have already set up your RealSense camera using the :doc:`RealSense setup tutorial `. If you have not, please set up the sensor and then restart this quickstart from the beginning. 2. Continuing inside the Docker container, install the following dependencies: :ir_apt: .. code:: bash sudo apt-get install -y ros-humble-isaac-ros-examples ros-humble-isaac-ros-realsense 3. Run the following launch file to spin up a demo of this package using RealSense camera: Resize: .. code:: bash ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=realsense_mono,resize Color Conversion: .. code:: bash ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=realsense_mono,color_conversion Crop: .. code:: bash ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=realsense_mono,crop Rectify: .. code:: bash ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=realsense_mono,rectify_mono Flip: .. code:: bash ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=realsense_mono,flip .. tab:: Hawk Camera #. Ensure that you have already set up your Hawk camera using the :doc:`Hawk setup tutorial `. If you have not, please set up the sensor and then restart this quickstart from the beginning. 2. Continuing inside the Docker container, install the following dependencies: :ir_apt: .. code:: bash sudo apt-get install -y ros-humble-isaac-ros-examples ros-humble-isaac-ros-argus-camera 3. Run the following launch file to spin up a demo of this package using Hawk: Resize: .. code:: bash ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=argus_mono,resize Color Conversion: .. code:: bash ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=argus_mono,color_conversion Crop: .. code:: bash ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=argus_mono,crop Rectify: .. code:: bash ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=argus_mono,rectify_mono Flip: .. code:: bash ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=argus_mono,flip .. tab:: ZED Camera #. Ensure that you have already set up your ZED camera using :doc:`ZED setup tutorial `. 2. Continuing inside the Docker container, install the following dependencies: :ir_apt: .. code:: bash sudo apt-get install -y ros-humble-isaac-ros-examples ros-humble-isaac-ros-zed 3. Run the following launch file to spin up a demo of this package using ZED camera: Resize: .. code:: bash ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=zed_mono,resize \ interface_specs_file:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_image_proc/zed2_quickstart_interface_specs.json Color Conversion: .. code:: bash ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=zed_mono,color_conversion \ interface_specs_file:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_image_proc/zed2_quickstart_interface_specs.json Crop: .. code:: bash ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=zed_mono,crop \ interface_specs_file:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_image_proc/zed2_quickstart_interface_specs.json Rectify: .. code:: bash ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=zed_mono,rectify_mono \ interface_specs_file:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_image_proc/zed2_quickstart_interface_specs.json Flip: .. code:: bash ros2 launch isaac_ros_examples isaac_ros_examples.launch.py launch_fragments:=zed_mono,flip \ interface_specs_file:=${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_image_proc/zed2_quickstart_interface_specs.json .. note:: If you are using the `ZED X` series, replace `zed2_quickstart_interface_specs.json` with `zedx_quickstart_interface_specs.json` in the above command. .. note:: For RealSense camera package issues, please refer to the section :ref:`here `. Other supported cameras can be found :ref:`here `. For camera calibration, please refer to :doc:`this guide `. Visualize Results ^^^^^^^^^^^^^^^^^ 1. Open a **new** terminal inside the Docker container: .. code:: bash cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \ ./scripts/run_dev.sh 2. Observe the image_proc output on the terminal with the command: Resize: .. code:: bash ros2 run image_view image_view --ros-args --remap image:=resize/image Color Conversion: .. code:: bash ros2 run image_view image_view --ros-args --remap image:=image_mono Crop: .. code:: bash ros2 run image_view image_view --ros-args --remap image:=crop/image Rectify: .. code:: bash ros2 run image_view image_view --ros-args --remap image:=image_rect Flip: .. code:: bash ros2 run image_view image_view --ros-args --remap image:=image_flipped Overview ^^^^^^^^ The ``isaac_ros_image_proc`` package offers functionality for rectifying/undistorting images from a monocular camera setup, resizing the image, changing the image format, and alpha blending two images. It largely replaces the ``image_proc`` package, though the image format conversion facility also functions as a way to replace the CPU-based image format conversion in ``cv_bridge``. The rectify node can also resize the image; if resizing is not needed, specify the output width/height same as input. Available Components ^^^^^^^^^^^^^^^^^^^^ AlphaBlendNode ~~~~~~~~~~~~~~ ROS Parameters ^^^^^^^^^^^^^^ .. list-table:: :header-rows: 1 * - ROS Parameter - Type - Default - Description * - ``alpha`` - ``double`` - ``0.5`` - The alpha value, where 0 represents the image input being fully transparent and the mask input fully opaque. This is a mandatory parameter and needs to be between 0 and 1 inclusive. * - ``image_queue_size`` - ``int`` - ``10`` - The queue size of the image input. This is a mandatory parameter and needs to be set to a non-zero positive number. * - ``mask_queue_size`` - ``int`` - ``10`` - The queue size of the mask input. This is a mandatory parameter and needs to be set to a non-zero positive number. * - ``sync_queue_size`` - ``int`` - ``10`` - The queue size of the time synchronizer. This is a mandatory parameter and needs to be set to a non-zero positive number. ROS Topics Subscribed ^^^^^^^^^^^^^^^^^^^^^ .. list-table:: :header-rows: 1 * - ROS Topic - Interface - Description * - ``image_input`` - :ir_repo:`NitrosImage ` - The input image to blend. * - ``mask_input`` - :ir_repo:`NitrosImage ` - The mask image to blend. ROS Topics Published ~~~~~~~~~~~~~~~~~~~~ .. list-table:: :header-rows: 1 * - ROS Topic - Interface - Description * - ``blended_image`` - :ir_repo:`NitrosImage ` - Blended image. CropNode ~~~~~~~~ ROS Parameters ^^^^^^^^^^^^^^ .. list-table:: :header-rows: 1 * - ROS Parameter - Type - Default - Description * - ``input_width`` - ``int64_t`` - ``0`` - The width of the input image. This is a mandatory parameter and needs to be set to a non-zero positive number. * - ``input_height`` - ``int64_t`` - ``0`` - The height of the input image. This is a mandatory parameter and needs to be set to a non-zero positive number. * - ``crop_width`` - ``int64_t`` - ``0`` - The width of the output image. This is a mandatory parameter and needs to be set to a non-zero positive number. * - ``crop_height`` - ``int64_t`` - ``0`` - The height of the output image. This is a mandatory parameter and needs to be set to a non-zero positive number. * - ``crop_mode`` - ``string`` - ``""`` - Mode of operation to perform cropping. Valid values are - ``CENTER``, ``LEFT``, ``RIGHT``, ``TOP``, ``BOTTOM``, ``TOPLEFT``, ``TOPRIGHT``, ``BOTTOMLEFT``, ``BOTTOMRIGHT``, and ``BBOX``. Except ``BBOX``, region of interest calculation is performed based on input and cropped image dimensions. * - ``roi_top_left_x`` - ``int64_t`` - ``0`` - X value of the top left corner of a region of interest. This is only used when ``crop_mode`` is set to ``BBOX``. * - ``roi_top_left_y`` - ``int64_t`` - ``0`` - Y value of the top left corner of a region of interest. This is only used when ``crop_mode`` is set to ``BBOX``. * - ``num_blocks`` - ``int`` - ``40`` - The number of pre-allocated memory output blocks, should not be less than ``40``. ROS Topics Subscribed ^^^^^^^^^^^^^^^^^^^^^ .. list-table:: :header-rows: 1 * - ROS Topic - Interface - Description * - ``image`` - :ir_repo:`NitrosImage ` - The input image that needs to be cropped. * - ``camera_info`` - :ir_repo:`NitrosCameraInfo ` - The corresponding camera_info of the input image. ROS Topics Published ~~~~~~~~~~~~~~~~~~~~ .. list-table:: :header-rows: 1 * - ROS Topic - Interface - Description * - ``crop/image`` - :ir_repo:`NitrosImage ` - Cropped image. * - ``crop/camera_info`` - :ir_repo:`NitrosCameraInfo ` - The corresponding camera_info of the cropped image. ImageFlipNode ~~~~~~~~~~~~~ ROS Parameters ^^^^^^^^^^^^^^ .. list-table:: :header-rows: 1 * - ROS Parameter - Type - Default - Description * - ``flip_mode`` - ``string`` - ``BOTH`` - Axis to be use for flip operation. Valid values are - ``HORIZONTAL``, ``VERTICAL``, and ``BOTH``. ROS Topics Subscribed ^^^^^^^^^^^^^^^^^^^^^ .. list-table:: :header-rows: 1 * - ROS Topic - Interface - Description * - ``image`` - :ir_repo:`NitrosImage ` - The input image that needs to be flipped. ROS Topics Published ~~~~~~~~~~~~~~~~~~~~ .. list-table:: :header-rows: 1 * - ROS Topic - Interface - Description * - ``image_flipped`` - :ir_repo:`NitrosImage ` - Flipped image. ImageFormatConverterNode ~~~~~~~~~~~~~~~~~~~~~~~~ ROS Parameters ^^^^^^^^^^^^^^ .. list-table:: :header-rows: 1 * - ROS Parameter - Type - Default - Description * - ``image_width`` - ``int64_t`` - ``1280`` - The width of the input image. This is a mandatory parameter and needs to be set to a non-zero positive number. * - ``image_height`` - ``int64_t`` - ``720`` - The height of the input image. This is a mandatory parameter and needs to be set to a non-zero positive number. * - ``encoding_desired`` - ``string`` - ``""`` - Target color encoding to convert to. Valid values are - ``img_encodings::RGB8``, ``img_encodings::RGB16``, ``img_encodings::BGR8``, ``img_encodings::BGR16``, ``img_encodings::MONO8``, ``img_encodings::MONO16``, ``img_encodings::NV24``, and ``nv12``. * - ``num_blocks`` - ``int`` - ``40`` - The number of pre-allocated memory output blocks, should not be less than ``40``. ROS Topics Subscribed ^^^^^^^^^^^^^^^^^^^^^ .. list-table:: :header-rows: 1 * - ROS Topic - Interface - Description * - ``image_raw`` - :ir_repo:`NitrosImage ` - The input image data. ROS Topics Published ~~~~~~~~~~~~~~~~~~~~ .. list-table:: :header-rows: 1 * - ROS Topic - Interface - Description * - ``image`` - :ir_repo:`NitrosImage ` - The output image data. ImageNormalizeNode ~~~~~~~~~~~~~~~~~~ ROS Parameters ^^^^^^^^^^^^^^ .. list-table:: :header-rows: 1 * - ROS Parameter - Type - Default - Description * - ``mean`` - ``std::vector`` - ``{0.5, 0.5, 0.5}`` - The mean of the image pixels per channel that will be used for normalization. * - ``stddev`` - ``std::vector`` - ``{0.5, 0.5, 0.5}`` - The standard deviation of the image pixels per channel that will be used for normalization. ROS Topics Subscribed ^^^^^^^^^^^^^^^^^^^^^ .. list-table:: :header-rows: 1 * - ROS Topic - Interface - Description * - ``image`` - :ir_repo:`NitrosImage ` - The input image data. ROS Topics Published ~~~~~~~~~~~~~~~~~~~~ .. list-table:: :header-rows: 1 * - ROS Topic - Interface - Description * - ``normalized_image`` - :ir_repo:`NitrosImage ` - The output image data. PadNode ~~~~~~~ ROS Parameters ^^^^^^^^^^^^^^ .. list-table:: :header-rows: 1 * - ROS Parameter - Type - Default - Description * - ``output_image_width`` - ``uint16_t`` - ``1200`` - The width of the output image. This is a mandatory parameter and needs to be set to a non-zero positive number. * - ``output_image_height`` - ``uint16_t`` - ``1024`` - The height of the output image. This is a mandatory parameter and needs to be set to a non-zero positive number. * - ``padding_type`` - ``string`` - ``CENTER`` - Describes the padding mode. Valid values are - ``CENTER``, ``TOP_LEFT``, ``TOP_RIGHT``, ``BOTTOM_LEFT``, and ``BOTTOM_RIGHT``. * - ``border_type`` - ``string`` - ``CONSTANT`` - Describes how border values are calculated. Valid values are - ``CONSTANT``, ``REPLICATE``, ``REFLECT``, ``WRAP``, and ``REFLECT101``. More info can be found `here `__. * - ``border_pixel_color_value`` - ``std::vector`` - ``{0.0, 0.0, 0.0, 0.0}`` - RGBA color value for border color. ROS Topics Subscribed ^^^^^^^^^^^^^^^^^^^^^ .. list-table:: :header-rows: 1 * - ROS Topic - Interface - Description * - ``image`` - :ir_repo:`NitrosImage ` - The input image data. ROS Topics Published ~~~~~~~~~~~~~~~~~~~~ .. list-table:: :header-rows: 1 * - ROS Topic - Interface - Description * - ``padded_image`` - :ir_repo:`NitrosImage ` - The output image data. RectifyNode ~~~~~~~~~~~ ROS Parameters ^^^^^^^^^^^^^^ .. list-table:: :header-rows: 1 * - ROS Parameter - Type - Default - Description * - ``output_width`` - ``uint16_t`` - ``1200`` - The width of the output image. This is a mandatory parameter and needs to be set to a non-zero positive number. * - ``output_height`` - ``uint16_t`` - ``1024`` - The height of the output image. This is a mandatory parameter and needs to be set to a non-zero positive number. * - ``horizontal_interval`` - ``uint16_t`` - ``1`` - The horizontal interval used for generating rectify warp grid points. It must be power-of-two and must be greater than or equal to 1. * - ``vertical_interval`` - ``uint16_t`` - ``1`` - The vertical interval used for generating rectify warp grid points. It must be power-of-two and must be greater than or equal to 1. * - ``interpolation`` - ``string`` - ``cubic_catmullrom`` - The interpolation type. Choices are: ``nearest``, ``linear``, ``cubic_catmullrom``. * - ``border_type`` - ``string`` - ``zero`` - The boarder type specifies how the pixel values outside of the image should be constructed. Choices are: ``zero``, ``repeat``, ``reverse``, ``mirror``. * - ``num_blocks`` - ``int`` - ``40`` - The number of pre-allocated memory output blocks, should not be less than ``40``. ROS Topics Subscribed ^^^^^^^^^^^^^^^^^^^^^ .. list-table:: :header-rows: 1 * - ROS Topic - Interface - Description * - ``image`` - :ir_repo:`NitrosImage ` - The input image that needs to be rectified. * - ``camera_info`` - :ir_repo:`NitrosCameraInfo ` - The corresponding camera_info of the input image. ROS Topics Published ~~~~~~~~~~~~~~~~~~~~ .. list-table:: :header-rows: 1 * - ROS Topic - Interface - Description * - ``crop/image`` - :ir_repo:`NitrosImage ` - Rectified image. * - ``crop/camera_info`` - :ir_repo:`NitrosCameraInfo ` - The corresponding camera_info of the rectified image. ResizeNode ~~~~~~~~~~ ROS Parameters ^^^^^^^^^^^^^^ .. list-table:: :header-rows: 1 * - ROS Parameter - Type - Default - Description * - ``input_width`` - ``int64_t`` - ``0`` - The width of the input image. This is a mandatory parameter and needs to be set to a non-zero positive number. * - ``input_height`` - ``int64_t`` - ``0`` - The height of the input image. This is a mandatory parameter and needs to be set to a non-zero positive number. * - ``output_width`` - ``int64_t`` - ``1080`` - The width of the output image. This is a mandatory parameter and needs to be set to a non-zero positive number. * - ``output_height`` - ``int64_t`` - ``720`` - The height of the output image. This is a mandatory parameter and needs to be set to a non-zero positive number. * - ``keep_aspect_ratio`` - ``bool`` - ``false`` - When enabled, aspect ratio of the original image will be preserved by adding black border. * - ``encoding_desired`` - ``string`` - ``""`` - This value is used to mark the preference of output color format. * - ``disable_padding`` - ``bool`` - ``false`` - When set to true, ``keep_aspect_ratio`` is ignored and no padding is added to the output image. * - ``num_blocks`` - ``int`` - ``40`` - The number of pre-allocated memory output blocks, should not be less than ``40``. ROS Topics Subscribed ^^^^^^^^^^^^^^^^^^^^^ .. list-table:: :header-rows: 1 * - ROS Topic - Interface - Description * - ``image`` - :ir_repo:`NitrosImage ` - The input image that needs to be resized. * - ``camera_info`` - :ir_repo:`NitrosCameraInfo ` - The corresponding camera_info of the input image. ROS Topics Published ~~~~~~~~~~~~~~~~~~~~ .. list-table:: :header-rows: 1 * - ROS Topic - Interface - Description * - ``resize/image`` - :ir_repo:`NitrosImage ` - Resized image. * - ``resize/camera_info`` - :ir_repo:`NitrosCameraInfo ` - The corresponding camera_info of the resized image. **Limitation**: ``isaac_ros_image_proc`` nodes require even number dimensions for images. .. |package_name| replace:: ``isaac_ros_image_proc``