Tutorial for cuVGL Map Creation

Introduction

The map creation process in cuVGL requires two inputs:

  • Raw Images

  • Poses

cuVGL extract features from the raw images and save them in the cuVGL map, along with the corresponding poses. The cuVGL map is structured as a folder that contains the following files:

  • keyframes: This folder contains the features extracted from the raw images, with each individual keyframe saved as a binary protobuf file.

  • keyframes/frames_meta.pb.txt: This protobuf file contains metadata for the keyframes including timestamp, poses, image_name, etc.

  • bow_index.pb: This is the bag-of-words index file for image retrieval.

  • vocabulary: This folder contains all the vocabulary files.

Map Creation With Isaac Perceptor

Use the Tutorial: Mapping and Localization on the Nova Carter to set up cuVGL with Isaac Perceptor on the robot.

Map Creation From ROS Bags

Warning

Map Creation, typically, uses all the CPU and GPU resources, make sure you do not have anything important running at the same time.

Note

Isaac Perceptor was tested with ROS 2 rosbags collected by a Nova Recorder isaac_ros_nova_recorder Quickstart.

If you want to create the cuVGL map from rosbags, follow these steps:

  1. Collect a rosbag with isaac_ros_nova_recorder Quickstart.

  2. Run your own SLAM algorithm to generate the poses. If you use cuVSLAM, see the Tutorial: Mapping and Localization on the Nova Carter to create the map instead.

  3. Extract features from the rosbags. You can use the following command to extract features from rosbags. cuVGL only supports converting h264 compressed image in the rosbag.

# Set following variables to your own paths
$SENSOR_DATA_BAG="path_to_sensor_data.bag"
$POSE_BAG="path_to_pose.bag"
$MAP_FOLDER="path_to_map_folder"

ros2 run isaac_mapping_ros run_rosbag_to_mapping_data.py \
--sensor_data_bag=$SENSOR_DATA_BAG --pose_bag=$POSE_BAG --output_folder="$MAP_FOLDER/keyframes" \
--extract_feature --rot_dist=5 --trans_dist=0.2 \
--pose_topic_name <POSE_TOPIC_NAME>

Note

Only geometry_msgs/msg/PoseStamped, geometry_msgs/msg/PoseWithCovarianceStamped, nav_msgs/msg/Odometry, and nav_msgs/msg/Path message types are supported for pose type in the pose rosbag.

  1. Create the global localization map using the following command:

    # Create the global localization map, it will create bow index, bow vocabulary
ros2 run isaac_mapping_ros create_cuvgl_map.py --map_folder=$MAP_FOLDER --no-extract_feature

  2. If you have a prebuilt vocabulary, copy the vocabulary folder or create symlink in the map folder, then pass --build_bow_voc=false when running create_cuvgl_map.py. For example:

ln -s /path/to/vocabulary $MAP_FOLDER/vocabulary
ros2 run isaac_mapping_ros create_cuvgl_map.py --map_folder=$MAP_FOLDER --no-extract_feature --no-build_bow_voc

Map Creation From Raw Images

While it’s recommended to directly use the rosbag data converter to create cuVGL data format, you can also create the map from raw images.

To do this you must prepare the keyframe frames_meta.pb.txt metadata file for your raw images.

The frames_meta.pb.txt is a text protobuf file of message KeyframesMetadataCollection, for the detailed definition see the keyframe_metadata.pb.h file under the install folder of cuVGL.

cat $(ros2 pkg prefix isaac_mapping_ros)/include/isaac_mapping/protos/visual/general/keyframe_metadata.pb.h

  1. Prepare a text protobuf file of message KeyframesMetadataCollection and put it under:

    $MAP_FOLDER/raw/
  image_0.jpg
  image_1.jpg
  ...
  frames_meta.pb.txt

  2. Create the global localization map using the following command:

    # Create the global localization map, it will create bow index, bow vocabulary
ros2 run isaac_mapping_ros create_cuvgl_map.py --map_folder=$MAP_FOLDER --no-extract_feature

  3. Run create map script with --extract_feature.