`isaac_manipulator_pick_and_place`#

Source code available on GitHub.

Overview#

isaac_manipulator_pick_and_place is the ROS 2 package implementation of the multi-object Pick and Place workflow, built on top of the isaac_manipulator_orchestration behavior tree framework. This package provides the executable nodes, launch files, and configuration templates needed to run the system on your robot hardware.

Package Contents:

ROS 2 Action Server: /multi_object_pick_and_place for workflow execution
Launch Files: Complete system orchestration with configurable parameters
Configuration Templates: YAML files for behavior tree and blackboard setup
Behavior Tree Nodes: Custom nodes for manipulation-specific operations
Integration Layer: Interfaces with Isaac ROS perception and motion packages

Quickstart#

Set Up Development Environment#

Set up your development environment by following the instructions in getting started.
(Optional) Install dependencies for any sensors you want to use by following the sensor-specific guides.

Note

We strongly recommend installing all sensor dependencies before starting any quickstarts. Some sensor dependencies require restarting the development environment during installation, which will interrupt the quickstart process.

Build `isaac_manipulator_pick_and_place`#

Binary Package

Activate the Isaac ROS environment:
```
isaac-ros activate
```

Install the prebuilt Debian package:

sudo apt-get update

sudo apt-get install -y ros-jazzy-isaac-manipulator-pick-and-place

Build from Source

Clone this repository under ${ISAAC_ROS_WS}/src:

cd ${ISAAC_ROS_WS}/src && \
   git clone --recursive -b release-4.0 https://github.com/NVIDIA-ISAAC-ROS/isaac_manipulator.git isaac_manipulator

Activate the Isaac ROS environment:
```
isaac-ros activate
```

Use rosdep to install the package’s dependencies:

sudo apt-get update

rosdep update && rosdep install --from-paths ${ISAAC_ROS_WS}/src/isaac_manipulator/isaac_manipulator_pick_and_place --ignore-src -y

Build the package from source:
```
cd ${ISAAC_ROS_WS} && \
   colcon build --symlink-install --packages-up-to isaac_manipulator_pick_and_place
```
Note

The --symlink-install flag allows you to edit configuration files in place without rebuilding. Changes to Python scripts and config files will be immediately available after sourcing.
Source the ROS workspace:

Note

Make sure to repeat this step in every terminal created inside the Isaac ROS environment.

Because this package was built from source, the enclosing workspace must be sourced for ROS to be able to find the package’s contents.
```
source install/setup.bash
```

Run Launch File#

This quickstart focuses on running the orchestration system standalone for development and testing.
Prepare configuration files:

Set up configuration files and environment variables based on your installation method:
Binary Package
Binary installations have read-only configuration files, so you must copy them to edit:
# Set up configuration directory and copy files mkdir -p ${ISAAC_ROS_WS}/isaac_manipulator_config cp $(ros2 pkg prefix --share isaac_manipulator_pick_and_place)/params/multi_object_pick_and_place_behavior_tree_params.yaml \ ${ISAAC_ROS_WS}/isaac_manipulator_config/multi_object_pick_and_place_behavior_tree_params.yaml cp $(ros2 pkg prefix --share isaac_manipulator_pick_and_place)/params/multi_object_pick_and_place_blackboard_params.yaml \ ${ISAAC_ROS_WS}/isaac_manipulator_config/multi_object_pick_and_place_blackboard_params.yaml
Files you’ll edit:
- Behavior tree parameters: ${ISAAC_ROS_WS}/isaac_manipulator_config/multi_object_pick_and_place_behavior_tree_params.yaml
- Blackboard parameters: ${ISAAC_ROS_WS}/isaac_manipulator_config/multi_object_pick_and_place_blackboard_params.yaml
# Point to the directory containing your configuration files export ISAAC_MANIPULATOR_PICK_AND_PLACE_CONFIG_DIR="${ISAAC_ROS_WS}/isaac_manipulator_config"
Build from Source
When building from source with --symlink-install, you can edit configuration files directly in the source directory. Changes take effect immediately after sourcing your workspace (no rebuild needed).

Files you’ll edit:
- Behavior tree parameters: ${ISAAC_ROS_WS}/src/isaac_manipulator/isaac_manipulator_pick_and_place/params/multi_object_pick_and_place_behavior_tree_params.yaml
- Blackboard parameters: ${ISAAC_ROS_WS}/src/isaac_manipulator/isaac_manipulator_pick_and_place/params/multi_object_pick_and_place_blackboard_params.yaml
# Point to the source directory containing the configuration files export ISAAC_MANIPULATOR_PICK_AND_PLACE_CONFIG_DIR="${ISAAC_ROS_WS}/src/isaac_manipulator/isaac_manipulator_pick_and_place/params"
For detailed configuration instructions, see the Configuration Guide.
Start the mock action servers and services for testing the orchestration logic:
```
ros2 launch isaac_manipulator_orchestration mock_servers.launch.py
```
Note

These are dummy/mock servers that provide the expected action and service interfaces but do not perform actual object detection, pose estimation, motion planning, or gripper control. They are only for testing the behavior tree orchestration logic.
(Optional) For interactive debugging and visualization:
```
py-trees-tree-viewer
```
Note

The py-trees-tree-viewer is a Qt-based graphical tool that provides real-time tree visualization, timeline replay, and blackboard inspection.

Launch the orchestration system:

ros2 launch isaac_manipulator_pick_and_place orchestration.launch.py \
   behavior_tree_config_file:=${ISAAC_MANIPULATOR_PICK_AND_PLACE_CONFIG_DIR}/multi_object_pick_and_place_behavior_tree_params.yaml \
   blackboard_config_file:=${ISAAC_MANIPULATOR_PICK_AND_PLACE_CONFIG_DIR}/multi_object_pick_and_place_blackboard_params.yaml \
   print_ascii_tree:=false

Send an action goal to test the behavior tree execution:

# Test with single bin mode
ros2 action send_goal --feedback /multi_object_pick_and_place isaac_manipulator_interfaces/action/MultiObjectPickAndPlace \
  '{target_poses: {header: {frame_id: "base_link"}, poses: [{position: {x: -0.25, y: 0.45, z: 0.50}, orientation: {w: 0.017994, x: -0.677772, y: 0.734752, z: 0.020993}}]}, class_ids: [], mode: 0}'

The behavior tree will execute with mock responses, allowing you to observe the orchestration logic and debug your configuration.

API and Configuration#

Related Resources:

Multi-Object Pick and Place Concepts - Architecture and workflow design
isaac_manipulator_orchestration - Behavior tree framework and orchestration details
Isaac for Manipulation Pick And Place Tutorial - Hardware workflow
Isaac Sim Tutorial - Simulation workflow

Integration#

This package integrates with the broader Isaac for Manipulation ecosystem and requires the following dependencies:

isaac_manipulator_orchestration: Behavior tree framework and shared behaviors
isaac_manipulator_servers: Action servers for perception and motion
isaac_manipulator_bringup: Complete system launch files

isaac_manipulator_pick_and_place#