Skip to content

Add Collect Training Data with Behaviors Objective to lab_sim - #849

Open
marioprats wants to merge 2 commits into
v10.0from
mp/lab-sim-collect-training-data-objective
Open

Add Collect Training Data with Behaviors Objective to lab_sim#849
marioprats wants to merge 2 commits into
v10.0from
mp/lab-sim-collect-training-data-objective

Conversation

@marioprats

@marioprats marioprats commented Aug 14, 2026

Copy link
Copy Markdown

[written by AI]

Closes PickNikRobotics/moveit_pro#21510

Problem

The Collect Training Data with Behaviors guide walks the reader through building a recording-wrapper Objective in lab_sim, but the Objective itself does not ship with the config — QA has to rebuild it by hand and users have no reference to open.

Changes

  • New Collect Training Data with Behaviors Objective (src/lab_sim/objectives/collect_training_data_with_behaviors.xml), matching the guide exactly: a Fallback whose first Sequence runs RecordEpisode (dataset_name=behavior_bottle_pick, config_name=lab_sim, num_episodes=1) → 2 s settle → the collapsed Pick April Tag Labeled Object Subtree → SaveEpisode → 1 s settle → StopRecordingConvertDataset. The second Sequence (StopRecordingAlwaysFailure) discards an unsaved failed attempt and preserves the failed result, so a failed pick is never labeled as a successful demonstration.
  • Pick from Pose: replaced the deprecated PlanCartesianPath position_only="false" port with the equivalent cartesian_constraint="[{constraint_type: position_and_orientation}]". The wrapped task uses this Subtree for its retract move, and the deprecated port made every run of the new Objective emit a deprecation warning.

Verification

  • All node IDs and port names were checked against the Behavior sources in moveit_pro (providedPorts() of RecordEpisode, SaveEpisode, StopRecording, ConvertDataset, PlanCartesianPath) rather than written from memory; the cartesian_constraint YAML form matches the parser's accepted syntax.
  • The Objective was run in lab_sim before the Pick from Pose port migration; that run surfaced the position_only deprecation warning this PR removes. It has not yet been re-run on top of the migration — worth one QA pass before merge.

Adds the recording-wrapper Objective from the Collect Training Data with
Behaviors how-to guide, and migrates Pick from Pose off the deprecated
PlanCartesianPath position_only port so the wrapped task runs without a
deprecation warning.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Aug 14, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 7b878371-35b8-453c-9c7e-25e802161b5d

📥 Commits

Reviewing files that changed from the base of the PR and between a7e40d7 and 30c7bec.

📒 Files selected for processing (1)
  • src/lab_sim/test/objectives_integration_test.py

📝 Walkthrough

Summary by CodeRabbit

  • New Features

    • Added a training-data collection workflow that records successful bottle-picking episodes and converts them into the LeRobotDataset v3.0 format.
    • Failed attempts are automatically discarded, while successful recordings are preserved.
    • Added recording and save-settlement delays to help ensure complete, reliable episodes.
  • Bug Fixes

    • Improved robot retraction planning by enforcing both position and orientation constraints for more consistent motion.

Walkthrough

Adds a lab simulation behavior tree for recording bottle-pick training data. Successful episodes are saved and converted to LeRobotDataset v3.0. Failed attempts are discarded. Retract planning now applies position and orientation constraints.

Changes

Training data collection

Layer / File(s) Summary
Retract path constraints
src/lab_sim/objectives/pick_from_pose.xml
The retract PlanCartesianPath action now uses explicit position and orientation Cartesian constraints.
Behavior-tree recording workflow
src/lab_sim/objectives/collect_training_data_with_behaviors.xml, src/lab_sim/test/objectives_integration_test.py
Adds recording setup, startup and save delays, the AprilTag bottle-pick task, successful episode saving, recording shutdown, dataset conversion, failed-attempt cleanup, and integration-test exclusion for the recording objective.

Possibly related PRs

Suggested reviewers: griswaldbrooks

Merge Risk: ⚪ Minimal · up to 30c7b

This PR adds the documented training-data objective and updates a deprecated planning port without any supplied merge-blocking correctness or operational risk; no actionable merge-blocking risk remains beyond normal checks and review.

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Description check ✅ Passed The description directly explains the new lab_sim Objective and the related PlanCartesianPath update.
Linked Issues check ✅ Passed The PR adds the Objective requested by issue #21510 and matches the described VLA data-collection workflow.
Out of Scope Changes check ✅ Passed The changes support the new Objective, including its deprecated-port migration and CI exclusion.
Human Review Check ✅ Passed The PR diff is limited to two lab_sim Objective XML changes and one integration-test skip; it adds no auth, infra, public API, tutorial, launch-script, or core behavior package changes.

Comment @coderabbitai help to get the list of available commands.

@marioprats
marioprats marked this pull request as ready for review August 14, 2026 10:20
coderabbitai[bot]
coderabbitai Bot previously approved these changes Aug 14, 2026
@github-actions

Copy link
Copy Markdown

MoveIt Pro Example WS - Objectives Integration Test Report

The objective requires data-collection setup not available on CI runners.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

MoveIt Pro Example WS - Objectives Integration Test Report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant