`isaac_ros_data_validation`

Source code on GitHub.

Overview

Tools to validate data from Nova. The isaac_ros_data_validation package primarily contains a utility to check ROSbags generated by isaac_ros_data_recorder for frame drops, desynchronization, and other issues.

Quickstart

Set Up Development Environment

Set up your development environment by following the instructions in getting started.

Clone isaac_ros_common under ${ISAAC_ROS_WS}/src.

cd ${ISAAC_ROS_WS}/src && \
   git clone -b release-3.1 https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common.git isaac_ros_common

(Optional) Install dependencies for any sensors you want to use by following the sensor-specific guides.

Warning

We strongly recommend installing all sensor dependencies before starting any quickstarts. Some sensor dependencies require restarting the Isaac ROS Dev container during installation, which will interrupt the quickstart process.

Download Quickstart Assets

Download quickstart data from NGC:

Make sure required libraries are installed.

sudo apt-get install -y curl jq tar

Then, run these commands to download the asset from NGC:

NGC_ORG="nvidia"
NGC_TEAM="isaac"
PACKAGE_NAME="isaac_ros_data_validation"
NGC_RESOURCE="isaac_ros_data_validation_assets"
NGC_FILENAME="quickstart.tar.gz"
MAJOR_VERSION=3
MINOR_VERSION=1
VERSION_REQ_URL="https://catalog.ngc.nvidia.com/api/resources/versions?orgName=$NGC_ORG&teamName=$NGC_TEAM&name=$NGC_RESOURCE&isPublic=true&pageNumber=0&pageSize=100&sortOrder=CREATED_DATE_DESC"
AVAILABLE_VERSIONS=$(curl -s \
    -H "Accept: application/json" "$VERSION_REQ_URL")
LATEST_VERSION_ID=$(echo $AVAILABLE_VERSIONS | jq -r "
    .recipeVersions[]
    | .versionId as \$v
    | \$v | select(test(\"^\\\\d+\\\\.\\\\d+\\\\.\\\\d+$\"))
    | split(\".\") | {major: .[0]|tonumber, minor: .[1]|tonumber, patch: .[2]|tonumber}
    | select(.major == $MAJOR_VERSION and .minor <= $MINOR_VERSION)
    | \$v
    " | sort -V | tail -n 1
)
if [ -z "$LATEST_VERSION_ID" ]; then
    echo "No corresponding version found for Isaac ROS $MAJOR_VERSION.$MINOR_VERSION"
    echo "Found versions:"
    echo $AVAILABLE_VERSIONS | jq -r '.recipeVersions[].versionId'
else
    mkdir -p ${ISAAC_ROS_WS}/isaac_ros_assets && \
    FILE_REQ_URL="https://api.ngc.nvidia.com/v2/resources/$NGC_ORG/$NGC_TEAM/$NGC_RESOURCE/\
versions/$LATEST_VERSION_ID/files/$NGC_FILENAME" && \
    curl -LO --request GET "${FILE_REQ_URL}" && \
    tar -xf ${NGC_FILENAME} -C ${ISAAC_ROS_WS}/isaac_ros_assets && \
    rm ${NGC_FILENAME}
fi

Build `isaac_ros_data_validation`

Launch the Docker container using the run_dev.sh script:

cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
./scripts/run_dev.sh

Install the prebuilt Debian package:

sudo apt-get install -y ros-humble-isaac-ros-data-validation

Clone this repository under ${ISAAC_ROS_WS}/src:

cd ${ISAAC_ROS_WS}/src && \
   git clone -b release-3.1 https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_nova.git isaac_ros_nova

Launch the Docker container using the run_dev.sh script:

cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
./scripts/run_dev.sh

Use rosdep to install the package’s dependencies:

rosdep install --from-paths ${ISAAC_ROS_WS}/src/isaac_ros_nova --ignore-src -y

Build the package from source:

cd ${ISAAC_ROS_WS} && \
   colcon build --symlink-install --packages-up-to isaac_ros_data_validation

Source the ROS workspace:

Note

Make sure to repeat this step in every terminal created inside the Docker container.

Because this package was built from source, the enclosing workspace must be sourced for ROS to be able to find the package’s contents.
```
source install/setup.bash
```

Run Launch File

Continuing inside the Docker container, run data validation:

python -m isaac_ros_data_validation.summarize_bag ${ISAAC_ROS_WS}/isaac_ros_assets/isaac_ros_data_validation/quickstart

This will generate the report:

======================================================== example.mcap ========================================================
Camera Drop Q Score: 100.0
Camera Bucket Q Score: 100.0
Intra Camera Sync Q score: 100.0

Camera Drop Table                          [................................................................]
/back_stereo_camera/left                   [................................................................]
/back_stereo_camera/right                  [................................................................]
/front_stereo_camera/left                  [................................................................]
/front_stereo_camera/right                 [................................................................]
/left_stereo_camera/left                   [................................................................]
/left_stereo_camera/right                  [................................................................]
/right_stereo_camera/left                  [................................................................]
/right_stereo_camera/right                 [................................................................]

Intra Camera Sync Table                    [................................................................]
/back_stereo_camera/camera_info            [................................................................]
/back_stereo_camera/image_compressed       [................................................................]
/front_stereo_camera/camera_info           [................................................................]
/front_stereo_camera/image_compressed      [................................................................]
/left_stereo_camera/camera_info            [................................................................]
/left_stereo_camera/image_compressed       [................................................................]
/right_stereo_camera/camera_info           [................................................................]
/right_stereo_camera/image_compressed      [................................................................]


Topics:
    /left_stereo_camera/left/camera_info | frames_captured: 301
    /back_stereo_camera/left/camera_info | frames_captured: 300
    /left_stereo_camera/right/camera_info | frames_captured: 301
    /back_stereo_camera/right/camera_info | frames_captured: 300
    /chassis/odom | frames_captured: 400
    /right_stereo_camera/left/image_compressed | frames_captured: 300
    /front_stereo_imu/imu | frames_captured: 969
    /right_stereo_camera/right/image_compressed | frames_captured: 300
    /front_stereo_camera/left/image_compressed | frames_captured: 300
    /chassis/ticks | frames_captured: 400
    /chassis/imu | frames_captured: 400
    /front_stereo_camera/right/image_compressed | frames_captured: 300
    /back_stereo_camera/left/image_compressed | frames_captured: 300
    /left_stereo_camera/left/image_compressed | frames_captured: 300
    /back_stereo_camera/right/image_compressed | frames_captured: 300
    /left_stereo_camera/right/image_compressed | frames_captured: 300
    /right_stereo_camera/left/camera_info | frames_captured: 300
    /right_stereo_camera/right/camera_info | frames_captured: 300
    /front_stereo_camera/left/camera_info | frames_captured: 300
    /front_stereo_camera/right/camera_info | frames_captured: 300
    /chassis/battery_state | frames_captured: 10


/back_stereo_camera/camera_info/sync:
    - Num Desyncs : 0
    - Percent Desynced: 0.0
    - Mean Difference : 0.0
    - Max Difference: [0.0]
    - Desync Table: ['................................................................']

/back_stereo_camera/image_compressed/sync:
    - Num Desyncs : 0
    - Percent Desynced: 0.0
    - Mean Difference : 0.0
    - Max Difference: [0.0]
    - Desync Table: ['................................................................']

/back_stereo_camera/left:
    - Percent Dropped: 0.0%
    - Number Dropped: 0
    - Mean Frequency: 30.00079422035675
    - Drop Table: [................................................................]
    - Frames Captured: 300
    - Total Jitter: 0.006412842809363813 ms
    - Jitter Excluding Drops: 0.006412842809363813 ms
    - Largest Frame Drop: 33.347072 ms

/back_stereo_camera/right:
    - Percent Dropped: 0.0%
    - Number Dropped: 0
    - Mean Frequency: 30.00079422035675
    - Drop Table: [................................................................]
    - Frames Captured: 300
    - Total Jitter: 0.006412842809363813 ms
    - Jitter Excluding Drops: 0.006412842809363813 ms
    - Largest Frame Drop: 33.347072 ms

/chassis/battery_state:
    - Percent Dropped/High-Jitter: (0.9,)%
    - Number Dropped/High-Jitter: 9
    - Mean Frequency: 0.9901003944077132
    - Drop / Jitter Table: [.......x......x......x......x......x......x......x......x......x]
    - Frames Captured: 10
    - Total Jitter: 976.6652543333333 ms
    - Jitter Excluding Drops: nan ms

/chassis/imu:
    - Percent Dropped/High-Jitter: (0.0,)%
    - Number Dropped/High-Jitter: 0
    - Mean Frequency: 40.000823066559676
    - Drop / Jitter Table: [................................................................]
    - Frames Captured: 400
    - Total Jitter: 8.333847739348368 ms
    - Jitter Excluding Drops: 8.333847739348368 ms

/chassis/odom:
    - Percent Dropped/High-Jitter: (0.0,)%
    - Number Dropped/High-Jitter: 0
    - Mean Frequency: 40.00078097564878
    - Drop / Jitter Table: [................................................................]
    - Frames Captured: 400
    - Total Jitter: 8.333821433583957 ms
    - Jitter Excluding Drops: 8.333821433583957 ms

/front_stereo_camera/camera_info/sync:
    - Num Desyncs : 0
    - Percent Desynced: 0.0
    - Mean Difference : 0.0
    - Max Difference: [0.0]
    - Desync Table: ['................................................................']

/front_stereo_camera/image_compressed/sync:
    - Num Desyncs : 0
    - Percent Desynced: 0.0
    - Mean Difference : 0.0
    - Max Difference: [0.0]
    - Desync Table: ['................................................................']

/front_stereo_camera/left:
    - Percent Dropped: 0.0%
    - Number Dropped: 0
    - Mean Frequency: 30.000731030522118
    - Drop Table: [................................................................]
    - Frames Captured: 300
    - Total Jitter: 0.006407705685617994 ms
    - Jitter Excluding Drops: 0.006407705685617994 ms
    - Largest Frame Drop: 33.349888 ms

/front_stereo_camera/right:
    - Percent Dropped: 0.0%
    - Number Dropped: 0
    - Mean Frequency: 30.000731030522118
    - Drop Table: [................................................................]
    - Frames Captured: 300
    - Total Jitter: 0.006407705685617994 ms
    - Jitter Excluding Drops: 0.006407705685617994 ms
    - Largest Frame Drop: 33.349888 ms

/left_stereo_camera/camera_info/sync:
    - Num Desyncs : 0
    - Percent Desynced: 0.0
    - Mean Difference : 0.0
    - Max Difference: [0.0]
    - Desync Table: ['................................................................']

/left_stereo_camera/image_compressed/sync:
    - Num Desyncs : 0
    - Percent Desynced: 0.0
    - Mean Difference : 0.0
    - Max Difference: [0.0]
    - Desync Table: ['................................................................']

/left_stereo_camera/left:
    - Percent Dropped: 0.0%
    - Number Dropped: 0
    - Mean Frequency: 30.000721783251574
    - Drop Table: [................................................................]
    - Frames Captured: 300
    - Total Jitter: 0.006984775919731731 ms
    - Jitter Excluding Drops: 0.006984775919731731 ms
    - Largest Frame Drop: 33.350144 ms

/left_stereo_camera/right:
    - Percent Dropped: 0.0%
    - Number Dropped: 0
    - Mean Frequency: 30.000721783251574
    - Drop Table: [................................................................]
    - Frames Captured: 300
    - Total Jitter: 0.006984775919731731 ms
    - Jitter Excluding Drops: 0.006984775919731731 ms
    - Largest Frame Drop: 33.350144 ms

/right_stereo_camera/camera_info/sync:
    - Num Desyncs : 0
    - Percent Desynced: 0.0
    - Mean Difference : 0.0
    - Max Difference: [0.0]
    - Desync Table: ['................................................................']

/right_stereo_camera/image_compressed/sync:
    - Num Desyncs : 0
    - Percent Desynced: 0.0
    - Mean Difference : 0.0
    - Max Difference: [0.0]
    - Desync Table: ['................................................................']

/right_stereo_camera/left:
    - Percent Dropped: 0.0%
    - Number Dropped: 0
    - Mean Frequency: 30.00076031358312
    - Drop Table: [................................................................]
    - Frames Captured: 300
    - Total Jitter: 0.006439670011147545 ms
    - Jitter Excluding Drops: 0.006439670011147545 ms
    - Largest Frame Drop: 33.347072 ms

/right_stereo_camera/right:
    - Percent Dropped: 0.0%
    - Number Dropped: 0
    - Mean Frequency: 30.00076031358312
    - Drop Table: [................................................................]
    - Frames Captured: 300
    - Total Jitter: 0.006439670011147545 ms
    - Jitter Excluding Drops: 0.006439670011147545 ms
    - Largest Frame Drop: 33.347072 ms

The Drop Q Score is simply 1 - (frames dropped / total frames). The Bucket Q Score is what percent of equally spaced buckets on the timeline have no data loss. In the owl-4 example we only drop 20% of frames, but because the data loss occurs throughout the recording the bucket Q score is only 2%.

The drop tables use an “X” table to show where in the recording a problem was found, in there is an issue with the batter_state not publishing at a constant rate.

If you want to start doing further analysis you can look at the isaac_ros_data_validation/example directory. read_mcap_example.py is a good place to start or ess_disparities_example.py if you have a db3 file.

Troubleshooting

Isaac ROS Troubleshooting

For solutions to problems with Isaac ROS, see troubleshooting.

API

Usage

python -m isaac_ros_data_validation.summarize_bag input_file

Command Line Arguments

Command Line Argument	Default Value	Description
`input_file`	None	Path to rosbag.
`-v/--verbosity`	`warning`	Verbosity level. Can be `dump`, `error`, `info`, `compact`, or `warning`.

isaac_ros_data_validation