You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
AppControlBench bsky re-run (2026-08-05, current main + #1587/#1601–#1606, gpt_high): three task verdicts lost to the same trap —
bsky-06 (fail): judge: "final screen is an image viewer/loading overlay with caption text, not the post detail/thread page"
bsky-07 (fail): judge: "final screen shows an image viewer with image options, not the Following feed with the overflow menu open"
bsky-18 (partial): judge: "image viewer/lightbox is open … but the image itself is not visible"
Pattern (also seen in the earlier 0.20.5 re-run analysis, previously unfiled): a tap intended for a post/thread row lands on the post's embedded image, Bluesky opens the full-screen lightbox, and from then on the agent's snapshots show viewer chrome while its thread-navigation taps hit the dimmed overlay. Models rarely recognize the modal and either keep tapping dead space or end the task there.
What's wanted
Snapshot should make the modal state unmistakable — the lightbox is a full-screen presentation layer; if the capture composition can mark it (like the system-modal probe does for alerts), the agent gets "you are in an image viewer, close button at …" instead of a confusing sparse tree.
Evidence
AppControlBench bsky re-run (2026-08-05, current main + #1587/#1601–#1606, gpt_high): three task verdicts lost to the same trap —
Pattern (also seen in the earlier 0.20.5 re-run analysis, previously unfiled): a tap intended for a post/thread row lands on the post's embedded image, Bluesky opens the full-screen lightbox, and from then on the agent's snapshots show viewer chrome while its thread-navigation taps hit the dimmed overlay. Models rarely recognize the modal and either keep tapping dead space or end the task there.
What's wanted
Repro: seeded bench feed (feedItem-by-whiskers.test has an image embed) — tap the post row near the image, then try to navigate to the thread.
Bench data:
data/gpt_high__agent-device/bsky-{06,07,18}in the AppControlBench repo (uncommitted re-run).