perf(prompt): restore the dropped "maximize parallel tool calls" guidance#750
Merged
Merged
Conversation
…ance Iteration 2 on the trajectory-step gap against the latest Claude Code (same model, same effort). Iteration 1 (#749) took the subset mean from 25.5 to 13.2 steps against CC's 7.5; this addresses the largest remaining structural difference. Measured on the same tasks: clawcodex emits more than one tool call per assistant turn in **5.7%** of steps, the latest Claude Code in **18.1%**. Every independent pair it does not batch is an extra step, so that ratio is most of what remains between the two step counts. Reading the trajectories side by side shows it plainly — on regex-log CC issued 8 calls, all Bash, batching probes ("for c in node deno perl ruby; do command -v $c; done") and creating files with heredocs; clawcodex issued 17, alternating Write then Bash and probing one command at a time. Cause is a truncated port. The reference bullet has three parts: You can call multiple tools in a single response. If you intend to call multiple tools and there are no dependencies between them, make all independent tool calls in parallel. Maximize use of parallel tool calls where possible to increase efficiency. However, if some tool calls depend on previous calls to inform dependent values, do NOT call these tools in parallel and instead call them sequentially. ... clawcodex kept only the first sentence. What was dropped is what turns a permission into a practice: the imperative to maximize, and the dependency carve-out. The carve-out matters as much as the push — without a statement of when NOT to parallelize, "call them in parallel" is risky advice a careful model will mostly decline, which is what the 5.7% looks like. Same defect class as #748 (dropped qualifiers on the file-creation bullet), and general rather than benchmark-specific: batching independent work is better agent behavior everywhere. Tests: 8843 passed. The new test pins all three clauses, since restoring only the push would leave the model without the safety half. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Iteration 2 on the trajectory-step gap. Reporting a null result on the mechanism — see Outcome.
The port gap (real)
Both dropped clauses matter: the imperative to maximize, and the dependency carve-out. Without a statement of when not to parallelize, "call them in parallel" is risky advice a careful model will mostly decline.
Why I targeted it
Measured on the same tasks, same model, same effort: clawcodex emits >1 tool call per assistant turn in 5.7% of steps against the latest Claude Code's 18.1%. Reading
regex-logside by side: CC issued 8 calls, all Bash, batching probes (for c in node deno perl ruby; do command -v $c; done) and creating files via heredoc; clawcodex issued 17, alternatingWrite→Bashand probing one command at a time.Outcome: correct fix, null mechanism
The parallel-call rate did not improve — 8.9% → 6.8% on the comparable tasks (noise at this n), still far from CC's 18.2%. Mean steps moved 13.0 → 12.7, inside noise.
I'm merging it anyway because the truncation is real and the restored text is what the reference actually says — but it should not be credited with a behavioral win, and the remaining step gap is not explained by batching.
Iteration progression (tasks clean in every run compared)
main)Per task at iter2 vs CC:
circuit-fibsqrt10 vs 13 (clawcodex now lower),fix-code-vulnerability12 vs 10,cancel-async-tasks11 vs 6,sparql-university14 vs 8,regex-log17 vs 9,distribution-search12 vs 5.Also notable: memory-file writes during tasks fell to 0 in this run, from 35 across the earlier 89-trial run.
What the residual is not
Two candidate causes I investigated and ruled out as harness divergences:
Write-then-Bashvs heredoc. clawcodex's "reserve Bash for system commands, prefer the dedicated tool" bullet is faithful to the reference — in fact softer than it (the reference adds "and only fallback on Bash if absolutely necessary"). The tool mix on the comparable set is close: Bash 71% vs 76%, Write 18% vs 13%.Tests: 8843 passed. The new test pins all three clauses, since restoring only the push would leave the model without the safety half.
🤖 Generated with Claude Code