Skip to content

Fix: lab_sim Tutorial 1 walk-through bugs#634

Open
rlpratt12 wants to merge 2 commits into
mainfrom
fix/lab-sim-tutorial-bugs
Open

Fix: lab_sim Tutorial 1 walk-through bugs#634
rlpratt12 wants to merge 2 commits into
mainfrom
fix/lab-sim-tutorial-bugs

Conversation

@rlpratt12
Copy link
Copy Markdown

@rlpratt12 rlpratt12 commented May 15, 2026

Summary

lab_sim Objective cleanups surfaced by a Tutorial 1 walk-through on 9.2.1.

Problem

  1. /masks_visualization doesn't auto-show. The ML Find Objects on Table Objective publishes masks but never calls SwitchUIPrimaryView, so a first-time user runs the Objective and sees nothing happen unless they know to manually switch a pane to /masks_visualization. The other ML Objectives in lab_sim (ml_segment_image, ml_find_bottles_on_table_from_image_exemplar) already auto-switch — this one is an outlier.
  2. Deeply nested redundant Sequence wrappers in pick_all_bottles_with_apriltags.xml. The Release and Retract subtree had a Sequence containing a Sequence containing a Sequence containing a Sequence with the actual actions. Each intermediate wrapper had exactly one child, so the nesting carries no behavior — but it is genuinely confusing when a tutorial reader opens the tree.
  3. Warm-up velocity_force_controller block at the top of Pick All Bottles with AprilTags. The objective starts with SwitchController → velocity_force_controller, zero_fts, SwitchController → joint_trajectory_admittance_controller before doing any picking, ostensibly to dodge a "cold start" on the first placement. The two controller swaps add a few seconds of dead time at the start of every run and the cost being guarded against is essentially zero — the same swap happens inside the place loop anyway.
  4. Another redundant Sequence wrapper around Clear Snapshot / Look at Table / Take Wrist Camera Snapshot. Same shape as the Release and Retract problem above, just in a different part of the same tree.

Approach

Two commits:

  1. fix(lab_sim): Tutorial 1 walk-through fixes
    • ml_find_objects_on_table.xml: append SwitchUIPrimaryView primary_view_name="/masks_visualization" to the top-level Sequence, mirroring the pattern used by the other ML Objectives in lab_sim.
    • pick_all_bottles_with_apriltags.xml: remove the three redundant Sequence wrappers inside Release and Retract. No reordering, no Behavior changes, just unwrap.
  2. fix(lab_sim): Remove warm-up block and redundant Sequence wrapper
    • pick_all_bottles_with_apriltags.xml: delete the entire warm-up Sequence (SwitchControllervelocity_force_controller, CallTriggerService /velocity_force_controller/zero_fts, SwitchController → joint_trajectory_admittance_controller).
    • pick_all_bottles_with_apriltags.xml: flatten the unnamed Sequence wrapping Clear Snapshot / Look at Table / Take Wrist Camera Snapshot into its parent.

Alternatives considered

  • Adding a docs note saying "manually switch your view pane" instead of fixing the Objective — pushes the friction onto every reader and is inconsistent with the other ML Objectives.
  • Keeping the warm-up block but moving it inside the loop's first iteration only — overcomplicates the tree for a problem that doesn't appear to exist in practice.
  • Auto-deduping unary-Sequence nesting via a pre-commit hook — would catch other Objectives that have similar redundant wrapping, but is out of scope for this fix.

Validation

  • pre-commit run --files src/lab_sim/objectives/ml_find_objects_on_table.xml src/lab_sim/objectives/pick_all_bottles_with_apriltags.xml — passes (prettier, Validate Objective favorites, codespell, mixed line endings, EOF).
  • XML well-formedness verified by prettier and python3 -c "import xml.etree.ElementTree as ET; ET.parse(...)" on both files.

Manual verification

  • Run ML Find Objects on Table in lab_sim and confirm the view pane swaps to /masks_visualization automatically once PublishMask2D returns.
  • Run Pick All Bottles with AprilTags and confirm: the picking loop still places bottles in the tray; no regression versus the previous behavior on the first place (the warm-up was supposedly there to make the first place not be a cold start — verify the first place still succeeds without it).

🤖 Generated with Claude Code

- ml_find_objects_on_table.xml: Add SwitchUIPrimaryView at the end of
  the tree so a view pane auto-switches to /masks_visualization once
  masks are published. Matches the behavior of the other ML Objectives
  in lab_sim (ml_segment_image,
  ml_find_bottles_on_table_from_image_exemplar) and restores the
  auto-show readers expect from Tutorial 1.
- pick_all_bottles_with_apriltags.xml: Flatten the "Release and Retract"
  subtree, which had three nested single-child Sequence wrappers around
  the actual actions. The wrappers were structurally inert but very
  confusing when a tutorial reader opens the tree.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@rlpratt12 rlpratt12 added this to the 9.4.0 milestone May 15, 2026
@rlpratt12 rlpratt12 requested a review from davetcoleman May 15, 2026 21:14
@rlpratt12 rlpratt12 self-assigned this May 15, 2026
@rlpratt12 rlpratt12 marked this pull request as ready for review May 15, 2026 21:16
- Drop the "Warm up velocity_force_controller" preamble (2x
  SwitchController + zero_fts). The cold-start cost it was guarding
  against is essentially zero in the current build, and the block adds
  ~3-4s of unnecessary controller-swap latency at the top of every run.
- Flatten the unnamed Sequence that wrapped Clear Snapshot / Look at
  Table / Take Wrist Camera Snapshot. The outer top-level Sequence
  already provides ordering, so the wrapper was inert and just made the
  tree harder to read.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@rlpratt12 rlpratt12 enabled auto-merge May 15, 2026 22:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant