fix(agent): re-introduce credential scrubbing on the tinyagents tool seam (tinyhumansai#4453) - #3
Closed
senamakel wants to merge 2 commits into
Closed
fix(agent): re-introduce credential scrubbing on the tinyagents tool seam (tinyhumansai#4453)#3senamakel wants to merge 2 commits into
senamakel wants to merge 2 commits into
Conversation
…seam (tinyhumansai#4453) The legacy engine scrubbed `scrub_credentials` over every tool output before it entered model context (v0.58.7 `engine/tools.rs`). The tinyagents migration dropped this: `execute_openhuman_tool` never scrubbed and `scrub_credentials` had zero production callers, so secrets in tool output (env dumps, config reads, API responses, shell output) reached model context, `session_raw` transcripts, worker-thread mirrors, and the `ToolCallOutcome` sink. Fix: - Re-introduce scrubbing as `CredentialScrubMiddleware`, a `wrap_tool` tool middleware installed on the shared `assemble_turn_harness` seam so BOTH the parent chat path and the sub-agent path are covered. It runs as the innermost wrap layer, so it always sees the RAW tool output and always runs BEFORE the `after_tool` chain (summarization / caps / tokenjuice) and the transcript push — independent of `after_tool` registration order (which tinyhumansai#4464 reworks). Because every downstream surface (transcript, ToolOutputMiddleware, the ToolCallOutcome sink, session_raw persistence, worker-thread mirror) derives from the scrubbed content, one scrub at this seam covers all of them. - Keep `harness/credentials.rs` as the single source of the patterns, and extend it beyond the legacy key/value regex to also catch the standalone secret formats the acceptance criteria name: bare AWS access-key IDs (`AKIA…`/`ASIA…`), OpenAI `sk-…` keys, and space-separated `Bearer` tokens. The scrub is idempotent. - Also scrub the tool result's `error` text (it can echo the raw output and is consumed by the outcome sink / model-facing error surface). Tests: - Unit tests in `credentials.rs` for the new AWS/OpenAI/Bearer patterns and idempotency. - e2e tests in `tinyagents/tests.rs` drive a real turn through `run_turn_via_tinyagents_shared` with a tool that leaks a fake `AKIA…`/`sk-…`/`Bearer` token and assert redaction at the model-visible transcript, the persisted conversation, and the `ToolCallOutcome` sink — on both the chat and the sub-agent path. Claude-Session: https://claude.ai/code/session_019j5TLsRLHsM3kqAYFyH4hR
… scrub The adapter-inventory test asserted 2 around-tool wraps; tinyhumansai#4453 adds CredentialScrubMiddleware as a 3rd (innermost) wrap, so the count is now 3. Caught by the workflow's static review (the original commit shipped with build_check=not_run and never ran this test). Ref: tinyhumansai#4453 Claude-Session: https://claude.ai/code/session_019j5TLsRLHsM3kqAYFyH4hR
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
Owner
Author
|
Superseded: re-implementing tinyhumansai#4453 fresh on current upstream/main as a PR to tinyhumansai/openhuman (upstream tinyhumansai#4483 changed the middleware area). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the credential-scrubbing regression from the TinyAgents 1.3 → 1.5 migration (source issue: tinyhumansai#4453).
The legacy engine ran
scrub_credentialsover every tool output before it entered model context (v0.58.7engine/tools.rs). On the tinyagents path,execute_openhuman_toolnever scrubbed andscrub_credentialshad zero production callers, so secrets in tool output (env dumps, config reads, API responses, shell output) reached model context (and thus third-party providers),session_rawtranscripts, worker-thread mirrors, and theToolCallOutcomesink — violating "Never log secrets or full PII."Fix
CredentialScrubMiddleware— awrap_tooltool middleware installed on the sharedassemble_turn_harnessseam, so both the parent chat path and the sub-agent path are covered. It is registered as the innermost wrap layer, so it always sees the RAW tool output and always runs BEFORE theafter_toolchain (summarization / caps / tokenjuice viaToolOutputMiddleware) and before the transcript push. This ordering holds regardless ofafter_toolregistration order — deliberately robust to the middleware-ordering rework in after_tool middleware order inverted: progressive-disclosure handoff is dead code, outcome capture sees pre-cap content tinyhumansai/openhuman#4464 (the crate runsafter_toolin reverse registration order; usingwrap_toolsidesteps that entirely). Because every downstream surface (transcript,ToolOutputMiddleware, theToolCallOutcomesink,session_rawpersistence viapersist_subagent_transcript, worker-thread mirror) derives from the scrubbed content, one scrub at this seam covers all of them.harness/credentials.rsstays the single source of truth for the patterns, extended beyond the legacy key/value regex to also catch the standalone formats the acceptance criteria name: bare AWS access-key IDs (AKIA…/ASIA…), OpenAIsk-…keys, and space-separatedBearertokens. The scrub is idempotent.errortext is scrubbed too (it can echo the raw output and feeds the outcome sink / model-facing error surface).Verified
credentials.rsfor the new AWS / OpenAI / Bearer patterns + idempotency (logic reviewed against each input; each raw secret is redacted, marker preserved).tinyagents/tests.rsdrive a real turn throughrun_turn_via_tinyagents_sharedwith a tool that leaks a fakeAKIA…/sk-…/Bearertoken and assert redaction at the model-visible transcript, the persisted conversation, and theToolCallOutcomesink — on both the chat and the sub-agent path.Could NOT verify locally
cargo check/cargo testdid not finish within the time budget: this worktree shares one Cargo target dir with several sibling worktree agents, and the build serialized on the shared target lock (20+ concurrentcargoprocesses). The change compiles against the current APIs as read from the vendored crate (ToolMiddleware/ToolHandler/MiddlewareToolOutcome,run_turn_via_tinyagents_shared,SubagentScope), but CI is the authoritative build/test signal here. Please rely on CI for the green check.https://claude.ai/code/session_019j5TLsRLHsM3kqAYFyH4hR