Reference Architecture#
The Isaac for Physical AI reference application captures teleoperated
demonstrations on the Unitree G1, uses them to fine-tune a GR00T N policy, and
deploys the fine-tuned policy back on the robot. The pipeline spans packages
shipped as part of Isaac ROS and also the
nvidia-isaac/gr00t-leapp-export
project (branch n17-export) for the
training steps.
Pipeline#
Teleoperate the Unitree G1 through an Extended Reality (XR) headset (package:
isaac_ros_unitree_g1_teleop_bringup). Refer to Teleoperate the Unitree G1.Record synchronized teleoperation episodes to MCAP rosbags (package:
isaac_ros_unitree_g1_recorder). Refer to Record Demonstrations.Convert the recorded session into a LeRobot dataset using the
mcap-to-lerobottool (package:isaac_ros_mcap_lerobot_converter). Refer to Convert MCAP Bags to a LeRobot Dataset.Fine-tune GR00T N on the LeRobot dataset (external — nvidia-isaac/gr00t-leapp-export, branch ``n17-export``). Refer to Fine-Tune GR00T N.
Export the fine-tuned policy using LEAPP (external — nvidia-isaac/gr00t-leapp-export, branch ``n17-export``). Refer to Export the Policy with LEAPP.
Deploy the exported policy on the robot (package:
isaac_ros_unitree_g1_gr00t). Refer to Deploy the Fine-Tuned Policy.
Component Map#
Component |
Role |
Reference |
|---|---|---|
|
Whole-body teleoperation for the robot |
|
|
Data recording |
|
|
Convert MCAP to LeRobot datasets |
|
|
Fine-tuning and LEAPP export |
|
|
Deploying the fine-tuned GR00T policy |
Supported Hardware#
Unitree G1 humanoid
Jetson AGX Thor, Jetson AGX Orin, or x86_64 systems with NVIDIA GPUs