feat(session): detect indirect posts wrapped in helper scripts in silent-exit classifier#80
Closed
dembrane-sam-bot wants to merge 1 commit into
Closed
feat(session): detect indirect posts wrapped in helper scripts in silent-exit classifier#80dembrane-sam-bot wants to merge 1 commit into
dembrane-sam-bot wants to merge 1 commit into
Conversation
…ent-exit classifier
Member
|
Closing — patches hole #2 (helper-script post) only. Hole #1 (bash-heredoc journal write) and hole #3 (cleanup bash after post) stay open, and hole #3 fired again on the session that opened this PR. Replacing the regex classifier with a structured |
auto-merge was automatically disabled
May 25, 2026 12:56
Pull request was closed
spashii
added a commit
that referenced
this pull request
May 25, 2026
…83) ## What this solves The recurring three-message cascade the operator has seen N times on Slack mentions: ``` msg 1 substantive reply ← what operator wanted msg 2 "apologies for the duplicate notify" ← false-positive retry msg 3 "<@op> something's wrong with me" ← daemon alert ``` Both layers were inferring "did Sam close the loop" from heuristics — regex over bash command strings (cascade layer 1) and the same inference re-applied to the retry session (cascade layer 2). Three known holes fired the cascade twice on 2026-05-25 in thread `1779688501.139669` (sessions `9843c66e6870` / `6afc1f60a477` / `a0ccae594a61` / `71e3df76652b`) and a third time on the bug-fix session `716d41decda1`. ## How Two commits, each independently revertable. ### `respond(text)` tool — `4a3542a` A structured close-the-loop tool on the main agent. Its call IS the gate signal — the classifier reads `respond_called`, not the trace. Cleanup work after the call (rm /tmp/x, tail /data/sessions.jsonl, journal writes) is harmless because ordering and substring matching no longer drive control flow. Auto-remediates the two mechanical mrkdwn rules: `### Heading` → `*Heading*`, standalone `---` → blank line. Warns (does not rewrite) on ALL CAPS labels and ` Sure!/Got it!` preambles — polish is OK, don't lock. Bash to ` chat.postMessage` still works as a fallback for experimental endpoints. The legacy regex classifier remains as the bash-path gate; ` respond` is canonical, not exclusive. Daily-maintenance flags recurring bash patterns that hit daemon/runtime limitations as tier-3 promotion candidates. ### Slack ground-truth alert suppression — `84c9f84` Before posting ` OPERATOR_ALERT_TEMPLATE`, query ` conversations.replies` on the originating thread. If the bot posted (via any path — including ones the regex missed), suppress the alert. The catch for genuine silent failures stays: when Slack confirms zero bot posts, the alert fires as today. Only affects the *alert* decision. The silent-exit retry itself still uses the regex classifier — that preserves the ACK-then-work-no-reply catch (sessions like ` 624e27ec` that opened PRs but never posted the result; Slack alone can't distinguish ACK from wrap-up). ## Cascade behavior, before vs after ``` BEFORE AFTER common case (clean post) 1 message 1 message bash hits regex hole 3 messages 2 messages both layers hit a hole 3 messages 2 messages genuine silent failure 3 messages 2 messages (the alert is the 2nd, which is correct here) ``` Worst case compresses from 3 → 2 permanently. ## What does NOT change - Voice rules in ` slack.md` / ` identity.md` — Sam's voice stays in Sam's prose. Only the two mechanical drift cases (` ###` headings, ` ---` rules) move to renderer behavior. - PR-comment followup — tracked in Linear separately. - Streaming — dropped from scope (separate question). - ` ask_operator` — unchanged. ## Tests 11 new tests in ` tests/runtime/test_silent_exit.py`: - 4 invariants for ` respond_called=True` paths (bash chaos around the call is harmless). - 2 fallback semantics (bash ` chat.postMessage` still satisfies; helper-script-without-respond still misses — documented limitation). - 5 for the alert-suppression helper + branch. All 181 tests pass locally. ## Files ``` src/capabilities/slack.md +18 -5 respond section, fallback rule, tier-3 link src/skills/daily-maintenance/skill.md +2 -0 tier-3 promotion signal note src/runtime/adk_runner.py +170 _clean_for_slack_mrkdwn + _make_respond_tool + registration src/runtime/prompts.py +24 -13 RETRY/SILENT_EXIT direct to `respond` src/runtime/session.py +32 -1 respond case in classifier, respond_called or fallback src/runtime/daemon.py +56 -2 _bot_posted_in_thread_since + suppression tests/runtime/test_silent_exit.py +335 -1 11 new tests, 4 helpers ``` ## Closes / supersedes Supersedes the closed ` #80` (helper-script regex patch) and ` #82` (prose rule asking the LLM to avoid the brittle classifier). ` #81` (sentence-case labels) stays merged unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is this change?
Detects indirect posts wrapped in helper scripts executed via
bashtool calls in the silent-exit classifier (session.py:910). When a script is executed via bash (e.g.python3 /tmp/post.py), the classifier looks back through the session's tool records to see if Sam wrote or edited that script file with a Slack post call (chat.postMessage/chat_postMessage/chat.update/chat_update). If so, it classifies the script execution as a loop-closing post.Adds deterministic unit tests covering script-based post detection and filename mismatch scenarios.
What did Sam notice that led to this?
Session
a0ccae594a61posted its final message by writing/tmp/post.pyand running it viapython3to handle complex multiline escaping. Because the stringchat.postMessageorchat.updatewas not literally present in the executed bash command (which waspython3 /tmp/post.py), the silent-exit classifier markedclosed_loop = False. This triggered a false silent-exit recovery and subsequent operator alert loop.Tier?
Tier 3 (runtime changes).
Confidence?
High confidence. Fully verified with all 24 unit tests passing deterministic evaluation.