`isaac_ros_manipulation_ros_python_utils`#

Source code available on GitHub.

Overview#

isaac_ros_manipulation_ros_python_utils is the foundational Python utility package for Isaac for Manipulation that provides core launch logic, configuration management, and integration utilities used across all Isaac for Manipulation workflows. This package works in conjunction with isaac_ros_manipulation_bringup and contains the essential functions that enable dynamic launch file generation, node creation, and system orchestration.

The package serves as the central library that isaac_ros_manipulation_bringup calls to instantiate and configure the Isaac for Manipulation system. All workflow types (Pose-to-Pose, Object Following, Pick and Place, Gear Assembly) depend on this package for their runtime configuration and execution.

Key Features#

Launch File Utilities: Dynamic node creation and launch configuration management
Configuration Management: Type-safe configuration classes with YAML parameter loading
Workflow Orchestration: High-level functions for creating workflow-specific node graphs
Perception Integration: Utilities for FoundationPose, RT-DETR, DOPE, and other perception pipelines
Robot Driver Integration: UR robot and Robotiq gripper configuration and setup
Geometry Utilities: Comprehensive ROS message geometry conversions and transformations
Type System: Strongly-typed enums and dataclasses for manipulator components
Planning Utilities: Joint space planning and IK solution selection helpers

Package Modules#

Configuration Module (`config.py`)#

Provides configuration classes that encapsulate all parameters needed for Isaac for Manipulation workflows.

Key Classes

Config: Base configuration class tracking use_sim_time
CameraConfig: Camera sensor configuration (RealSense)
DepthConfig: Depth estimation configuration (ESS, FoundationStereo)
PoseEstimationConfig: Pose estimation configuration (FoundationPose, DOPE)
ObjectDetectionConfig: Object detection configuration (RT-DETR)
ObjectSelectionConfig: Object selection and filtering configuration
WorkflowConfig: Workflow-specific configuration (Pose-to-Pose, Pick and Place, etc.)
GearAssemblyConfig: Gear assembly workflow configuration
OrchestrationConfig: Behavior tree orchestration configuration
CoreConfig: Master configuration aggregating all component configurations.

Key Functions

def load_yaml_params(
    params_file_path: str,
    package_name: str = 'isaac_ros_manipulation_bringup'
) -> Dict

Loads and processes YAML configuration files with support for command substitution. This function is called by isaac_ros_manipulation_bringup to load workflow configurations from YAML files.

Core Module (`core.py`)#

Contains core orchestration functions for creating workflow nodes and system containers.

Key Functions

def get_workflow_nodes(core_config: CoreConfig) -> List[Node]

Returns the appropriate workflow nodes based on the configured workflow type. This is the primary entry point called by isaac_ros_manipulation_bringup to instantiate workflow-specific nodes.

def get_manipulation_container(core_config: CoreConfig) -> Node

Creates a composable node container that runs multiple manipulation nodes in a single process for performance optimization.

def get_cumotion_node(
    core_config: CoreConfig,
    moveit_config: MoveItConfigsBuilder
) -> Node

Creates the cuMotion planning node with appropriate configuration for GPU-accelerated motion planning.

def get_nvblox_node(
    camera_type: str,
    use_sim_time: bool,
    num_cameras: int
) -> Node

Creates nvblox node for real-time 3D reconstruction and ESDF generation.

Workflows Module (`workflows.py`)#

Provides workflow-specific node creation functions.

Key Functions

def get_pose_to_pose() -> List[Node]

Returns nodes for the Pose-to-Pose reference workflow.

def get_object_following() -> List[Node]

Returns nodes for the Object Following reference workflow.

def get_pick_and_place_orchestrator(
    workflow_config: GearAssemblyConfig
) -> List[Node]

Returns the Pick and Place orchestrator node with appropriate configuration. Used by Gear Assembly workflow.

def get_multi_object_pick_and_place(
    orchestration_config: OrchestrationConfig
) -> List[Union[ExecuteProcess]]

Returns the multi-object Pick and Place behavior tree orchestration process.

Perception Module (`perception.py`)#

Handles perception pipeline setup for object detection and pose estimation.

Key Functions

def get_foundation_pose_nodes(
    camera_config: CameraConfig,
    workflow_type: WorkflowType,
    pose_estimation_config: PoseEstimationConfig
) -> List[Node]

Creates FoundationPose perception pipeline nodes including tracking and object detection integration.

def get_dope_nodes(
    camera_config: CameraConfig,
    dope_config: DopeConfig,
    workflow_type: WorkflowType
) -> List[Node]

Creates DOPE perception pipeline nodes for pose estimation.

def get_object_detection_servers(
    camera_config: CameraConfig,
    pose_estimation_config: PoseEstimationConfig
) -> List[Node]

Creates object detection server nodes (typically RT-DETR based).

def get_object_selection_server(
    object_selection_config: ObjectSelectionConfig
) -> List[Node]

Creates object selection server node for filtering and selecting detected objects.

Drivers Module#

Per-vendor driver setup lives in each robot’s *_driver_utils package. Every vendor exposes a class that extends isaac_ros_manipulation_robot_utils.robot_controller_base.RobotControllerBase (URDriverUtils for UR, FlexivDriverUtils for Flexiv) and implements the three launch-time methods the framework expects: get_robot_state_publisher, get_moveit_group_node, get_robot_control_nodes. Vendor-specific helpers such as get_isaac_sim_joint_parser_node, start_tool_communication, and get_grav_gripper_node remain module-level functions on the same package.

Instantiate the vendor class once in your driver launch file and call the methods directly:

from isaac_ros_manipulation_ur_driver_utils import (
    get_isaac_sim_joint_parser_node, start_tool_communication,
    URDriverUtils, UrRobotiqDriverConfig,
)

def launch_setup(context, *args, **kwargs):
    driver_config = UrRobotiqDriverConfig(context)
    ur = URDriverUtils(driver_config)

    nodes = [
        ur.get_robot_state_publisher(),
        *ur.get_robot_control_nodes(),
    ]
    move_group_node, moveit_config = ur.get_moveit_group_node()
    nodes.append(move_group_node)
    return nodes

Sensors Module (`sensors.py`)#

Handles sensor setup and calibration.

Key Functions

def get_calibration_parameters(
    workflow_type: WorkflowType,
    camera_type: CameraType,
    num_cameras: int
) -> List[Dict]

Returns camera calibration parameters for the specified setup.

def get_camera_nodes(
    num_cameras: str,
    setup: str,
    camera_config: CameraConfig
) -> List[Node]

Creates camera driver nodes (RealSense) with appropriate configuration.

Robot Description Utilities#

URDF/SRDF generation helpers (get_robot_description_contents_for_real_robot, get_robot_description_contents_for_sim) are now per-vendor and live alongside the driver code: isaac_ros_manipulation_ur_driver_utils.robot_description for UR and isaac_ros_manipulation_flexiv_driver_utils.robot_description for Flexiv. Import the vendor-specific module directly, for example:

from isaac_ros_manipulation_ur_driver_utils.robot_description import (
    get_robot_description_contents_for_real_robot,
    get_robot_description_contents_for_sim,
)

Geometry Module (`geometry.py`)#

Comprehensive geometry utilities for ROS message manipulation and coordinate transformations.

Key Conversion Functions

list_to_vector3(), vector3_to_list()
list_to_point(), point_to_list()
list_to_quaternion(), quaternion_to_list()
list_to_pose(), pose_to_list()
quaternion_to_rpy(), rpy_to_quaternion()
quaternion_to_rotation_matrix(), rotation_matrix_to_quaternion()
transform_to_matrix(), matrix_to_transform()

Key Transformation Functions

quaternion_multiply(): Quaternion composition
quaternion_difference(): Relative rotation
pose_difference(): Relative pose
compute_transformation_delta(): Transformation difference with metrics
rotate_pose(): Apply rotation to pose

Planning Utilities (`planning_utils.py`)#

Utilities for motion planning and IK solution selection.

Key Functions

def find_closest_joint_state(
    target_joint_position: np.ndarray,
    ik_possible_joint_states: List[JointState]
) -> JointState

Finds the closest joint state to a target configuration from IK solutions.

def get_sorted_indexes_of_closest_joint_states(
    ik_possible_joint_states: List[JointState],
    target_joint_position: np.ndarray
) -> List[int]

Returns sorted indices of joint states by distance to target.

Launch Utilities (`launch_utils.py`)#

Helper functions for launch file processing and type conversions.

Key Functions

get_str_variable(): Extract string from LaunchContext
get_float_variable(): Extract float from LaunchContext
get_bool_variable(): Extract boolean from LaunchContext
get_camera_type(): Parse camera type enum
get_workflow_type(): Parse workflow type enum
get_pose_estimation_type(): Parse pose estimation type enum
get_object_detection_type(): Parse object detection type enum
extract_pose_from_parameter(): Convert list to Pose message
extract_vector3_from_parameter(): Convert list to Vector3 message

Manipulator Types (`manipulator_types.py`)#

Defines strongly-typed enums and constants used throughout Isaac for Manipulation.

Key Enums

WorkflowType: POSE_TO_POSE, OBJECT_FOLLOWING, PICK_AND_PLACE, GEAR_ASSEMBLY
CameraType: REALSENSE
DepthType: ESS_FULL, ESS_LIGHT, REALSENSE, ISAAC_SIM, FOUNDATION_STEREO_LOW_RES, FOUNDATION_STEREO_HIGH_RES
PoseEstimationType: FOUNDATION_POSE, DOPE
ObjectDetectionType: RT_DETR, DOPE, GROUNDING_DINO, SEGMENT_ANYTHING, SEGMENT_ANYTHING2
SegmentationType: SAM, SEGMENT_ANYTHING_2
ObjectSelectionType: POINT_CLOUD_CLUSTERING
ObjectAttachmentShape: CUBOID, MESH
GripperType: ROBOTIQ_2F_85, ROBOTIQ_2F_140, ROBOTIQ_HAND_E
TrackingType: VISUAL_SLAM, REALSENSE_T265

Gear Assembly Module (`gear_assembly.py`)#

Specialized utilities for the Gear Assembly reference workflow.

Key Functions

def get_gear_assembly_nodes(
    gear_assembly_config: GearAssemblyConfig,
    use_sim_time: bool
) -> List[Node]

Creates gear assembly support nodes (RL policy server, goal pose publisher, insertion status checker).

def get_gear_assembly_orchestrator(
    gear_assembly_config: GearAssemblyConfig
) -> List[Node]

Creates the gear assembly orchestrator node with complete configuration.

def parse_joint_state_from_yaml(
    yaml_file_path: str,
    use_sim_time: bool
) -> JointState

Parses joint state configurations from YAML files.

Grasp Reader Module (`grasp_reader.py`)#

Utilities for reading and processing grasp configurations.

Key Functions

def lookup_transform_from_tf(
    tf_buffer,
    target_frame: str,
    source_frame: str
) -> Transform

Looks up transformations from TF tree with timeout handling.

Test Utilities (`test_utils.py`)#

Utilities for testing Isaac for Manipulation components.

Key Functions

def get_params_from_config_file_set_in_env(
    run_test: bool
) -> Dict

Loads test configuration from environment-specified file.

Constants Module (`constants.py`)#

Defines system-wide constants including:

Default file paths
Topic names
Action server names
Frame IDs
Timeout values
Package resource paths

Integration with `isaac_ros_manipulation_bringup`#

isaac_ros_manipulation_ros_python_utils is the core dependency of isaac_ros_manipulation_bringup. The bringup package’s launch files follow this pattern:

1. Load Configuration

from isaac_ros_manipulation_ros_python_utils.config import load_yaml_params

params = load_yaml_params('workflow_config.yaml')

2. Create Configuration Objects

from isaac_ros_manipulation_ros_python_utils import CoreConfig

core_config = CoreConfig(context)

3. Generate Nodes

from isaac_ros_manipulation_ros_python_utils import get_workflow_nodes

workflow_nodes = get_workflow_nodes(core_config)

4. Return Launch Description

The bringup package orchestrates these utilities to create complete launch descriptions for all supported workflows.

Usage Examples#

Example 1: Loading Workflow Configuration#

from isaac_ros_manipulation_ros_python_utils.config import load_yaml_params

# Load workflow configuration
params = load_yaml_params(
    params_file_path='pick_and_place_config.yaml',
    package_name='isaac_ros_manipulation_bringup'
)

# Access configuration values
use_sim_time = params['use_sim_time']
workflow_type = params['workflow_type']

Example 2: Creating Workflow Nodes#

from isaac_ros_manipulation_ros_python_utils import CoreConfig, get_workflow_nodes
from launch import LaunchContext

# Create configuration
core_config = CoreConfig(launch_context)

# Get workflow-specific nodes
workflow_nodes = get_workflow_nodes(core_config)

Example 3: Using Geometry Utilities#

from isaac_ros_manipulation_ros_python_utils.geometry import (
    list_to_pose, pose_to_list, quaternion_to_rpy, rotate_pose
)
from geometry_msgs.msg import Pose

# Convert list to Pose message
pose = list_to_pose([0.5, 0.3, 0.2, 0.0, 0.0, 0.0, 1.0])

# Rotate pose around Z axis
rotated_pose = rotate_pose(pose, angle=1.57, axis='z')

# Get RPY from quaternion
rpy = quaternion_to_rpy(pose.orientation)

Example 4: Creating Perception Nodes#

from isaac_ros_manipulation_ros_python_utils.perception import (
    get_foundation_pose_nodes, get_object_detection_servers
)
from isaac_ros_manipulation_ros_python_utils.manipulator_types import WorkflowType

# Create FoundationPose nodes
fp_nodes = get_foundation_pose_nodes(
    camera_config=camera_config,
    workflow_type=WorkflowType.PICK_AND_PLACE,
    pose_estimation_config=pose_estimation_config
)

# Create object detection servers
detection_nodes = get_object_detection_servers(
    camera_config=camera_config,
    pose_estimation_config=pose_estimation_config
)

Example 5: Type-Safe Configuration#

from isaac_ros_manipulation_ros_python_utils.manipulator_types import (
    CameraType, WorkflowType, PoseEstimationType
)
from isaac_ros_manipulation_ros_python_utils.config import WorkflowConfig

# Use type-safe enums
config = WorkflowConfig(launch_context)
config.camera_type = CameraType.REALSENSE
config.workflow_type = WorkflowType.PICK_AND_PLACE
config.pose_estimation_type = PoseEstimationType.FOUNDATION_POSE

Dependencies#

The package depends on several core ROS 2 and Isaac ROS components:

Core Dependencies

geometry_msgs: ROS geometry message definitions
isaac_ros_launch_utils: Isaac ROS launch utilities
isaac_ros_manipulation_interfaces: Action and service definitions

Build Dependencies

isaac_ros_common: Isaac ROS common infrastructure

Test Dependencies

ament_copyright: Copyright linting
ament_flake8: Python style checking
ament_pep257: Docstring linting
python3-pytest: Python testing framework
isaac_ros_test: Isaac ROS test utilities

Architecture Notes#

Design Philosophy

isaac_ros_manipulation_ros_python_utils follows a functional programming approach with configuration objects as parameters. This design:

Enables easy testing with mock configurations
Allows dynamic node creation based on runtime parameters
Maintains separation of concerns between launch logic and node implementation
Provides type safety through Python type hints and enums

Performance Considerations

Configuration parsing is done once at launch time
Node creation functions are lightweight and return quickly
Heavy computation is delegated to launched ROS nodes
The manipulation container allows multiple nodes to run in a single process

Extensibility

The modular design makes it easy to:

Add new workflow types
Support additional camera types
Integrate new perception algorithms
Customize robot configurations

isaac_ros_manipulation_ros_python_utils#

Overview#

Key Features#

Package Modules#

Configuration Module (config.py)#

Core Module (core.py)#

Workflows Module (workflows.py)#

Perception Module (perception.py)#

Drivers Module#

Sensors Module (sensors.py)#

Robot Description Utilities#

Geometry Module (geometry.py)#

Planning Utilities (planning_utils.py)#

Launch Utilities (launch_utils.py)#

Manipulator Types (manipulator_types.py)#

Gear Assembly Module (gear_assembly.py)#

Grasp Reader Module (grasp_reader.py)#

Test Utilities (test_utils.py)#

Constants Module (constants.py)#

Integration with isaac_ros_manipulation_bringup#

Usage Examples#

Example 1: Loading Workflow Configuration#

Example 2: Creating Workflow Nodes#

Example 3: Using Geometry Utilities#

Example 4: Creating Perception Nodes#

Example 5: Type-Safe Configuration#

Dependencies#

Architecture Notes#

See Also#

`isaac_ros_manipulation_ros_python_utils`#

Configuration Module (`config.py`)#

Core Module (`core.py`)#

Workflows Module (`workflows.py`)#

Perception Module (`perception.py`)#

Sensors Module (`sensors.py`)#

Geometry Module (`geometry.py`)#

Planning Utilities (`planning_utils.py`)#

Launch Utilities (`launch_utils.py`)#

Manipulator Types (`manipulator_types.py`)#

Gear Assembly Module (`gear_assembly.py`)#

Grasp Reader Module (`grasp_reader.py`)#

Test Utilities (`test_utils.py`)#

Constants Module (`constants.py`)#

Integration with `isaac_ros_manipulation_bringup`#