feat(method): orientation-honesty — honest WM1 ≤12 via harness isolation + self-explaining surfaces (4 tasks)#155
Merged
Conversation
…sted WM workspace resolves its own project
orientation-honesty task 1 of 5. The call-residuals pre-measure anatomy
(benchmark/results/2026-07-callres-preflight-anatomy.md) proved 7-13 startup
cmds/rep were HARNESS-induced: the benchmark runs each WM workspace at
benchmark/runs/<arm>/wm<N>/workspace — INSIDE AIDD-Book's own .add/ tree — so
the agent's first pre-init `status` walks up and resolves the PARENT project,
then burns ~10 cmds grepping find_root/STATE_FILE internals before `init`. A
real greenfield user never hits this; the number was partly metering the nesting.
- add_engine/io_state.py :: find_root gains an OPT-IN env ceiling. When
`ADD_ROOT_CEILING` is set, the upward walk stops at that dir (INCLUSIVE — the
break is AFTER the state.json check so the workspace's own post-init .add/
still resolves) and never ascends above it. Both cwd and ceiling are
.resolve()d so a symlinked tmpdir (macOS /var->/private/var) still matches.
Env UNSET is byte-identical to the legacy walk — the guard is invisible to
every non-harness caller (mirrors the engine's ADD_NO_UPDATE_CHECK/NO_COLOR
opt-in idiom). Synced x4 engine twins.
- benchmark/runner/core.py :: _invoke_once passes
env={**os.environ, "ADD_ROOT_CEILING": str(cwd)} to the agent Popen, so every
benchmark run scopes the agent's root-walk to its workspace. cmd_init already
writes .add/ at cwd, so only the pre-init READ leaked — one env injection fixes it.
- ENGINE_PKG_MD5 -> 955023db (add_engine package digest re-pinned, x4 twins);
ENGINE_MD5 UNCHANGED (add.py untouched); SEAMS untouched (no add.py line moved).
Red first: test_findroot_ceiling (engine — ancestor-above-ceiling resolved the
PARENT before the code) + benchmark/tests/test_workspace_isolation (env UNSET).
Green: 28 engine (incl. test_engine_extract_md5 PKG parity + test_engine_extract_io_state
+ test_engine_repin_parity) + 189 benchmark + full tooling fence. One human freeze
at the frozen §3 contract.
author: Tin Dang <tindang.ht97@gmail.com>
…t it resolved + hands the exact init orientation-honesty task 2 of 5. Behind the harness ceiling (task 1), a real user in a nested/monorepo dir with no local .add/ still gets `status` silently resolving an ANCESTOR project — the same confusion the pre-measure anatomy saw the benchmark agent grep find_root internals over. Now `status` says so and hands the fix. - new _ancestor_note(): when cwd has no .add/state.json of its own but find_root() walked up to one, returns a one-line note naming the resolved ancestor + the exact `add.py init --name "<project>" --stage <...>` to scope a project HERE (the _require_root skip-error precedent — hand the command with flags, never a bare hint). Silent when cwd owns a project or none is reachable. - cmd_status prints it to STDERR on the FULL path only — --json/--brief/--section stay silent so machine callers and pipes are never polluted; exit code + the normal status body are unchanged. Message-layer only — no gate/enforcement/state change. Red first (test_status_ancestor_warn: note-present + two silent cases). Synced x4 add.py twins, ENGINE_MD5 -> 9476543399; SEAMS _declared_scope re-pinned 5688->5711 (cmd_status grew _ancestor_note above it). Full fence green (3604 passed). author: Tin Dang <tindang.ht97@gmail.com>
…low map, not the 50-choice dump orientation-honesty task 3 of 5. The pre-measure anatomy showed the flag-carrying next-command hints from prior tasks (first-call-ergonomics / skip-error-ergonomics / engine-hint-batch-ops) already cover freeze/advance/new-task/init — the sixphase agent actually ran `freeze --by ... --cross`. The true residual is the stubborn 1/rep INITIAL orientation probe: `add.py --help` (and bare `add.py`) dumped the ~50-choice argparse usage, which help-habit-kill never touched (it only caught UNKNOWN commands). - new _FLOW_MAP: a flow-ordered map — status (start here) · init · new-task · advance · freeze · gate — each with its key flags on one line. - _AddArgParser.format_help(): the TOP parser (prog == "add.py") LEADS with the map, then the full argparse command list still follows — nothing lost; `add.py --help | head` now surfaces orientation instead of alphabet soup. A subcommand's own --help (prog "add.py <cmd>") is byte-identical argparse. - _AddArgParser.error(): a bare `add.py` (no subcommand → "the following arguments are required") prints the map + `run: add.py status` to stderr, exit 2 — not the raw dump. The help-habit-kill invalid-choice branch is unchanged. Message-layer only — no gate/enforcement/state change. Red first (test_orient_map: map-leads-then-full-list · bare-to-stderr · subcommand-help-unchanged · unknown- command-unchanged). Synced x4 add.py twins, ENGINE_MD5 -> a88bc24c; SEAMS unchanged (_declared_scope still 5711 — the class sits below it). Full fence green (3608 passed). author: Tin Dang <tindang.ht97@gmail.com>
…guide` (guide-fold)
The anatomy's orientation residual: agents re-run `add.py guide` after every
advance even though the `next:` footer already carries the phase command + short
why. The ONE line the footer lacks is the guide's chapter reference — so a
completing `advance` now folds it in.
A completing advance that lands an agent INTO a working phase prints, right ABOVE
the `next:` footer:
guide: .add/docs/<chapter> — the phase chapter (this + the next line ARE
`add.py guide`; no separate call)
The chapter is the LANDED phase's (PHASE_GUIDE[nxt][1], the same source
cmd_guide's `read:` line uses), so specify→plan folds 05-step-3-plan.md,
plan→tests folds 06-step-4-tests.md, etc. This is strictly non-duplicative: the
footer keeps its command+why unchanged (still exactly one `next:` line); the fold
adds only the chapter + the don't-re-run cue.
Suppressed where it would be wrong:
- landing in `done` folds nothing — Arm B (the milestone decision) owns that
juncture, only the footer prints.
- bundle fast-forward intermediates never fold (they `return` above the print);
only the final landing folds — exactly one fold per completing advance.
- fail-soft: a corrupt/unmapped landed phase folds nothing via PHASE_GUIDE.get,
never a KeyError on an already-saved advance (the footer's own ethos).
Kept in cmd_advance (not inside _next_footer) so the footer stays a pure
single-`next:`-line resolver reused by status/gate/new-task — the fold must not
leak a chapter line onto those surfaces.
Task 4/5 of the orientation-honesty milestone (lever E). Red/green:
test_guide_fold — the 3 fold asserts were RED (0 fold lines) before the print
existed; the done-landing assert guards the fold from the Arm-B juncture. Synced
×4 add.py twins; ENGINE_MD5→c8e0a3e5; SEAMS _declared_scope re-pinned 5711→5723
(the fold + comment sit above it in cmd_advance).
author: Tin Dang <tindang.ht97@gmail.com>
…e-shipped Record the milestone outcome: 4 built tasks (harness-workspace-isolation, status-ancestor-warn, orient-map, guide-fold) all gate=PASS + green, and lever C (scope-violation-explain) found ALREADY SHIPPED by the prior scope-gate-repair-path work (M1 add.py:1487 prevention, M2 add.py:5907 repair — both explain the rule + emit the paste-ready `re-cross --by`). No redundant 5th task built (human decision 2026-07-14). The milestone GOAL (honest WM1 <=12) stays open on the deferred, human-gated paid re-measure in the isolated harness. author: Tin Dang <tindang.ht97@gmail.com>
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
| import os | ||
| import shutil | ||
| import tempfile | ||
| import unittest |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
orientation-honesty — honest WM1 ≤12 via harness isolation + self-explaining surfaces
Follow-on to call-residuals (PR #154). After that milestone, the user asked to investigate the real runs before spending on the paid WM1 re-measure. I anatomized the actual
sixphase-r{1,2,3}/wm1transcripts (benchmark/results/2026-07-callres-preflight-anatomy.md), which reshaped the plan:--helpflag-discovery probes + 15.7 real per rep..add/tree, so the agent's first pre-initstatusroot-walked UP to the parent project. A real greenfield user never hits this — part of the metric was measuring the nesting, not the method.What shipped (4 tasks)
find_rootgains opt-inADD_ROOT_CEILING(bounded upward walk); the WM runner sets it to the workspace dir so each rep resolves its OWN projecta87ed1estatusprints a stderr note + exactinitcommand when cwd has no.add/but an ancestor resolved (--json/--briefsilent)1d9a211add.py/--helpLEAD with a concise flow map (status/init/new-task/advance/freeze/gate) then the full list — kills the 50-choice dump probe. Pivoted here because lever B's flagged-next-command form was already shipped0798bd2advancefolds the LANDED phase's guide chapter (guide: .add/docs/<chapter>+ "no re-runadd.py guide" cue) above the footer — non-duplicative (footer already carries command+why)91b3371What did NOT ship — lever C (scope-violation-explain) was already shipped
Investigation found the
scope_violationself-explaining surface already exists from the priorscope-gate-repair-pathwork:add.py:1487) — warns on the template-default §5 Scope + namesre-cross --by.add.py:5907) — explains the full resolution rule + emits the paste-readyadd.py re-cross --by <name>+ the revert alternative.The code comment at
add.py:5902-5905records this exact fix already killed "the live-benchmark agent source-diving for ~10 turns to discover re-cross" — the precise pain lever C was scoped to fix. Per First-Principles honesty, no redundant 5th task was built (human decision 2026-07-14 — close the build at 4).Discipline
_declared_scopemigrated 5688→5711→5723..pyccompiled into the bundle by the test run itself, removed — not a code defect).Deferred (human-gated)
The milestone GOAL — an honest WM1 ≤12 — stays open on the paid re-measure IN THE ISOLATED HARNESS (calls ≤12, fidelity ≥0.97, ≤1
--help/rep, zero root-walk confusion). That spend is the user's to run after this merges. The benchmark model pin (claude-sonnet-5) is unchanged for reproducibility.