============== |package_name| ============== :ir_github:` ` Quickstart ---------- 1. Set up your development environment by following the instructions :doc:`here `. 2. Clone ``isaac_ros_common`` and this repository under ``${ISAAC_ROS_WS}/src``. .. code:: bash cd ${ISAAC_ROS_WS}/src .. code:: bash git clone :ir_clone:`` .. code:: bash git clone :ir_clone:`` 3. Pull down a ROS Bag of sample data: .. code:: bash cd ${ISAAC_ROS_WS}/src/isaac_ros_object_detection/isaac_ros_detectnet && \ git lfs pull -X "" -I "resources/rosbags" 4. Launch the Docker container using the ``run_dev.sh`` script: .. code:: bash cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \ ./scripts/run_dev.sh 5. Install this package's dependencies. :ir_apt: .. code:: bash sudo apt-get install -y ros-humble-isaac-ros-detectnet ros-humble-isaac-ros-triton ros-humble-isaac-ros-dnn-image-encoder 6. Run the quickstart setup script which will download the `PeopleNet Model `__ from NVIDIA GPU Cloud(NGC) .. code:: bash cd /workspaces/isaac_ros-dev/src/isaac_ros_object_detection/isaac_ros_detectnet && \ ./scripts/setup_model.sh --height 632 --width 1200 --config-file resources/quickstart_config.pbtxt 7. Run the following launch file to spin up a demo of this package: .. code:: bash cd /workspaces/isaac_ros-dev && \ ros2 launch isaac_ros_detectnet isaac_ros_detectnet_quickstart.launch.py 8. Visualize and validate the output of the package in the ``rqt_image_view`` window. After about a minute, your output should look like this: .. figure:: :ir_lfs:`` :alt: RQT showing detection of people :align: center Try More Examples ----------------- To continue your exploration, check out the following suggested examples: .. toctree:: :maxdepth: 1 Tutorial with Isaac Sim Tutorial with Custom Model This package only supports models based on the ``Detectnet_v2`` architecture. Some of the `supported DetectNet models `__ from NGC: ============================================================================================= ====================================================== Model Name Use Case ============================================================================================= ====================================================== `PeopleNet AMR `__ People counting with a mobile robot `PeopleNet `__ People counting, heatmap generation, social distancing `TrafficCamNet `__ Detect and track cars `DashCamNet `__ Identify objects from a moving object `FaceDetectIR `__ Detect faces in a dark environment with IR camera ============================================================================================= ====================================================== To learn how to use this models, click :doc:`here `. ROS 2 Graph Configuration ------------------------- To run the DetectNet object detection inference, the following ROS 2 nodes must be set up and running: .. figure:: :ir_lfs:`` :width: 800px :align: center 1. **Isaac ROS DNN Image encoder**: This takes an image message and converts it to a tensor (``TensorList`` :ir_repo:`isaac_ros_tensor_list_interfaces/TensorList `) that can be processed by the network. 2. **Isaac ROS DNN Inference - Triton**: This executes the DetectNet network and takes, as input, the tensor from the DNN Image Encoder. .. note:: The :ir_repo:`Isaac ROS TensorRT ` package is not able to perform inference with DetectNet models at this time. The output is a TensorList message containing the encoded detections. Use the parameters ``model_name`` and ``model_repository_paths`` to point to the model folder and set the model name. The ``.plan`` file should be located at ``$model_repository_path/$model_name/1/model.plan`` 3. **Isaac ROS Detectnet Decoder**: This node takes the TensorList with encoded detections as input, and outputs ``Detection2DArray`` messages for each frame. See the following section for the parameters. Troubleshooting --------------- Isaac ROS Troubleshooting ~~~~~~~~~~~~~~~~~~~~~~~~~ For solutions to problems with Isaac ROS, see :doc:`troubleshooting `. Deep Learning Troubleshooting ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ For solutions to problems with using DNN models, see :doc:`troubleshooting deeplearning `. API ---- Usage ~~~~~ .. code:: bash ros2 launch isaac_ros_detectnet isaac_ros_detectnet.launch.py label_list:= enable_confidence_threshold:= enable_bbox_area_threshold:= enable_dbscan_clustering:= confidence_threshold:= min_bbox_area:= dbscan_confidence_threshold:= dbscan_eps:= dbscan_min_boxes:= dbscan_enable_athr_filter:= dbscan_threshold_athr:= dbscan_clustering_algorithm:= bounding_box_scale:= bounding_box_offset:= ROS Parameters ~~~~~~~~~~~~~~ =============================== ============ ============================= =========================================================================================================================================================================================================================== ROS Parameter Type Default Description =============================== ============ ============================= =========================================================================================================================================================================================================================== ``label_list`` ``string[]`` ``{"person", "bag", "face"}`` The list of labels. These are loaded from ``labels.txt`` (downloaded with the model) ``confidence_threshold`` ``double`` ``0.35`` The min value of confidence used to threshold detections before clustering ``min_bbox_area`` ``double`` ``100`` The min value of bounding box area used to threshold detections before clustering ``dbscan_confidence_threshold`` ``double`` ``0.35`` Holds the epsilon to control merging of overlapping boxes. Refer to OpenCV ``groupRectangles`` and DBSCAN documentation for more information on epsilon. ``dbscan_eps`` ``double`` ``0.7`` Holds the epsilon to control merging of overlapping boxes. Refer to OpenCV ``groupRectangles`` and DBSCAN documentation for more information on epsilon. ``dbscan_min_boxes`` ``int`` ``1`` The minimum number of boxes to return. ``dbscan_enable_athr_filter`` ``int`` ``0`` Enables the area-to-hit ratio (ATHR) filter. The ATHR is calculated as: **ATHR = sqrt(clusterArea) / nObjectsInCluster.** ``dbscan_threshold_athr`` ``double`` ``0.0`` The ``area-to-hit`` ratio threshold. ``dbscan_clustering_algorithm`` ``int`` ``1`` The clustering algorithm selection. (``1``: Enables DBScan clustering, ``2``: Enables Hybrid clustering, resulting in more boxes that will need to be processed with NMS or other means of reducing overlapping detections. ``bounding_box_scale`` ``double`` ``35.0`` The scale parameter, which should match the training configuration. ``bounding_box_offset`` ``double`` ``0.0`` Bounding box offset for both X and Y dimensions. =============================== ============ ============================= =========================================================================================================================================================================================================================== ROS Topics Subscribed ~~~~~~~~~~~~~~~~~~~~~ ============== ==================================================================================================================================================================== =============================================================== ROS Topic Interface Description ============== ==================================================================================================================================================================== =============================================================== ``tensor_sub`` :ir_repo:`isaac_ros_tensor_list_interfaces/TensorList ` The tensor that represents the inferred aligned bounding boxes. ============== ==================================================================================================================================================================== =============================================================== ROS Topics Published ~~~~~~~~~~~~~~~~~~~~ ======================== =============================================================================================================================== ================================================== ROS Topic Interface Description ======================== =============================================================================================================================== ================================================== ``detectnet/detections`` `vision_msgs/Detection2DArray `__ Aligned image bounding boxes with detection class. ======================== =============================================================================================================================== ================================================== .. |package_name| replace:: ``isaac_ros_detectnet``