Reference Architecture#

The Isaac for Physical AI reference application captures teleoperated demonstrations on the Unitree G1, uses them to fine-tune a GR00T N policy, and deploys the fine-tuned policy back on the robot. The pipeline spans packages shipped as part of Isaac ROS and also the nvidia-isaac/gr00t-leapp-export project (branch n17-export) for the training steps.

Pipeline#

  1. Teleoperate the Unitree G1 via an XR headset (package: isaac_ros_unitree_g1_teleop_bringup). See Teleoperate the Unitree G1.

  2. Record synchronized teleoperation episodes to MCAP rosbags (package: isaac_ros_unitree_g1_recorder). See Record Demonstrations.

  3. Convert the recorded session into a LeRobot dataset using the mcap-to-lerobot tool (package: isaac_ros_mcap_lerobot_converter). See Convert MCAP Bags to a LeRobot Dataset.

  4. Fine-tune GR00T N on the LeRobot dataset (external — nvidia-isaac/gr00t-leapp-export, branch ``n17-export``). See Fine-Tune GR00T N.

  5. Export the fine-tuned policy using LEAPP (external — nvidia-isaac/gr00t-leapp-export, branch ``n17-export``). See Export the Policy with LEAPP.

  6. Deploy the exported policy on the robot (package: isaac_ros_unitree_g1_gr00t). See Deploy the Fine-Tuned Policy.

Component Map#

Component

Role

Reference

isaac_ros_unitree_g1_teleop_bringup

Whole-body teleoperation for the robot

isaac_ros_unitree_g1_teleop_bringup

isaac_ros_unitree_g1_recorder

Data recording

isaac_ros_unitree_g1_recorder

isaac_ros_mcap_lerobot_converter

Convert MCAP to LeRobot datasets

converter README on GitHub

nvidia-isaac/gr00t-leapp-export

Fine-tuning and LEAPP export

finetune_new_embodiment.md

isaac_ros_unitree_g1_gr00t

Deploying the fine-tuned GR00T policy

isaac_ros_unitree_g1_gr00t

Supported Hardware#

  • Unitree G1 humanoid

  • Jetson AGX Thor or x86_64 systems with NVIDIA GPUs