Set Up a Simulated Robot#

Set up Isaac Sim, build the source workspace, and prepare the Flexiv Rizon 4s with the Grav gripper for DisplayPort pickup and task-space policy insertion.

Before You Begin#

First complete Workspace and Developer Environment Setup on your simulation workstation. This shared prerequisite creates ${ISAAC_ROS_WS}/src and prepares the environment used by the source builds below.

Review System Requirements. The workflow requires at least 25 GB GPU VRAM and 30 GB CPU RAM. Also review the Isaac Sim 6.0 requirements.

The source-build steps are included below. This simulation uses configured poses, so you need the FoundationPose and Segment Anything (SAM) model setup only for the physical-robot perception workflow.

Isaac Sim Setup#

Install Isaac Sim 6.0 using the native workstation installation instructions. The DisplayPort tutorial uses this version with its published scene from the Isaac Sim 6.1 asset path.

Use the bundled Jazzy ROS 2 libraries in a fresh simulator terminal, separate from the Isaac ROS source workspace terminal. Set the installation path to your Isaac Sim directory and configure the bridge:

export ISAAC_SIM_PACKAGE_PATH=${HOME}/isaacsim-6.0.0 # set to the path of your Isaac Sim package
export ROS_DISTRO=jazzy
export ROS_DOMAIN_ID=231
export RMW_IMPLEMENTATION=rmw_fastrtps_cpp
export FASTDDS_BUILTIN_TRANSPORTS=UDPv4
export LD_LIBRARY_PATH="${LD_LIBRARY_PATH:-}:${ISAAC_SIM_PACKAGE_PATH}/exts/isaacsim.ros2.core/${ROS_DISTRO}/lib"

Refer to the Isaac Sim ROS 2 bridge instructions for details. The DisplayPort launcher opens the required scene and starts the bridge. Follow the launch section below after building the workspace.

Build the Simulation Workspace#

Use a separate Isaac ROS terminal for the following steps. Set Fast DDS in every ROS terminal and use the same ROS_DOMAIN_ID as the simulator (231 in the example above).

Configure Fast DDS#

Isaac Sim uses Fast DDS for its ROS 2 bridge. After activating the Isaac ROS environment, set the following in every terminal that you use to build or run the simulation:

source /opt/ros/lyrical/setup.bash
export RMW_IMPLEMENTATION=rmw_fastrtps_cpp

Build Isaac ROS Manipulation Packages#

If you already built these packages following the package installation guide, skip the repeated build steps below and continue with Build Topic-Based Control.

Build the manipulation packages and their Isaac ROS dependencies from source. Place the dependency repositories in ${ISAAC_ROS_WS}/src before running rosdep so --ignore-src resolves those packages from the workspace. The dependency installation commands below provide system and ROS prerequisites.

  1. Clone the isaac_ros_manipulation repository under ${ISAAC_ROS_WS}/src:

    cd ${ISAAC_ROS_WS}/src && git clone --recursive -b release-r1 https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_manipulation.git isaac_ros_manipulation

  2. Activate the Isaac ROS environment:

    isaac-ros activate

  3. Install system dependencies:

    sudo apt update && sudo apt install -y python3-pip python3-git && pip install --break-system-packages pyvers==0.2.2

  4. Use rosdep to install the package’s dependencies:

    sudo apt-get update
rosdep update && rosdep install --from-paths ${ISAAC_ROS_WS}/src/isaac_ros_manipulation/isaac_ros_manipulation_bringup --ignore-src -y

  5. Build the package from source, with automatic asset installation disabled. Prepare any perception models required by your workflow separately.

    cd ${ISAAC_ROS_WS}
ISAAC_ROS_SKIP_ASSET_INSTALL=1 MANIPULATOR_INSTALL_ASSETS=0 colcon build --symlink-install --packages-up-to isaac_ros_manipulation_bringup

  6. Source the ROS workspace and set your ROS_DOMAIN_ID:

    Note

    Repeat this step in every terminal that you open inside the Isaac ROS environment.

    Because you built this package from source, you must source the enclosing workspace so that ROS can find the package’s contents.

    source install/setup.bash
export ROS_DOMAIN_ID=<ID_NUMBER>

Build Topic-Based Control#

NVIDIA maintains a fork of the original topic_based_ros2_control repository with the changes needed to support the impedance controller that the sim-to-real workflows require.

  1. Clone the main branch of this repository under ${ISAAC_ROS_WS}/src:

    cd ${ISAAC_ROS_WS}/src && \
  git clone --branch main --single-branch \
    https://github.com/NVIDIA-ISAAC-ROS/topic_based_ros2_control

  2. Install dependencies:

    rosdep update && rosdep install --from-paths ${ISAAC_ROS_WS}/src/topic_based_ros2_control --ignore-src -y

  3. Build the package from source.

    Set Fast DDS before invoking colcon so CMake selects an installed RMW implementation while configuring the package.

    cd ${ISAAC_ROS_WS} && \
  export RMW_IMPLEMENTATION=rmw_fastrtps_cpp && \
  ISAAC_ROS_SKIP_ASSET_INSTALL=1 MANIPULATOR_INSTALL_ASSETS=0 colcon build --symlink-install \
    --packages-select topic_based_ros2_control && \
  source install/setup.bash

  4. Confirm that ROS resolves the source workspace:

    ros2 pkg prefix topic_based_ros2_control

    The command should print a path under ${ISAAC_ROS_WS}/install. Source ${ISAAC_ROS_WS}/install/setup.bash in every terminal used to launch this workflow.

Note

If the driver nodes still fail to spawn, check your ros2 control package versions as described in isaac_ros_manipulation issue #18. This issue occurs because ros2 control updates can break application binary interface (ABI) compatibility. Make sure all ros2 control packages in the ros2_control repository are at the same version.

Prepare Configuration Files and Environment Variables#

Edit the configuration files in your source workspace and set up the environment variables below.

  1. When you build from source with --symlink-install, you can edit configuration files directly in the source directories.

  2. Set up environment variables:

    # Point to the source directories containing the configuration files
export ISAAC_ROS_MANIPULATION_WORKFLOW_CONFIG_DIR="${ISAAC_ROS_WS}/src/isaac_ros_manipulation/isaac_ros_manipulation_bringup/params"

Run Your First Simulated Workflow#

Launch the DisplayPort Scene#

Follow DisplayPort task-space policy pickup and insertion to obtain the launcher, load the DisplayPort scene, and start the simulator. Use the same ROS_DOMAIN_ID and middleware configuration in every ROS terminal.

Simulation Checks#

After starting the DisplayPort scene and ROS components, follow the simulation checks to verify the clock and robot state topics. Use the DisplayPort tutorial’s troubleshooting steps if its action server or policy topics are unavailable.