docs: record the test-secret convention in the project guidelines - #6658
docs: record the test-secret convention in the project guidelines#6658mzxchandra wants to merge 1 commit into
Conversation
Secret scanning reads every commit in a PR, not the final tree, so a fixture that merely looks like a real credential fails CI and keeps failing until the commit that introduced it is rewritten out of the branch history. That is an expensive lesson to learn per-person, and it was learned the expensive way. Writes down what the existing tests already do - a named PLACEHOLDER_ constant with an obviously-fake value and the TSDoc line seven test files already carry - and the part that was not written down anywhere: never build a fixture out of a real credential's structure, because a JWT header segment or a plausible base64 blob reads as a live secret to the scanner. The paragraph describes those shapes rather than quoting one, since a document warning about credential-shaped literals is a poor place to leave one.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
PR SummaryLow Risk Overview The new text explains that secret scanning evaluates every commit in a PR, not only the final tree, so removing a bad fixture later still leaves CI red until that commit is rewritten out of branch history. It standardizes naming and TSDoc for obvious placeholders (e.g. It also adds the missing rule: do not build fixtures from real credential structure (JWT header segments, common key prefixes, plausible base64 blobs). When structure matters for the test, keep only what the test exercises and make the rest unmistakably fake—and the doc describes those shapes without embedding literal examples. Reviewed by Cursor Bugbot for commit c3fe57d. Bugbot is set up for automated code reviews on this repo. Configure here. |
Greptile SummaryAdds project guidance for credential-shaped test fixtures to prevent false-positive secret-scanning failures.
Confidence Score: 5/5The documentation-only change appears safe to merge. The added guidance matches established placeholder patterns and introduces no runtime, build, security, or public-contract failure.
|
| Filename | Overview |
|---|---|
| CLAUDE.md | Documentation-only addition accurately records safe test-secret fixture conventions without changing runtime behavior. |
Reviews (1): Last reviewed commit: "docs: record the test-secret convention ..." | Re-trigger Greptile
1 participant
Summary
Records the convention this repo's tests already follow for credential-shaped fixtures, plus the part that was not written down anywhere.
Split out of #6650, where it was unrelated to the connector work.
Why
Secret scanning reads every commit in a PR, not the final tree. A fixture that merely looks like a real credential fails CI, and it keeps failing after you remove it — because the commit that introduced it is still in the branch history. Clearing it costs a history rewrite and a force-push.
That happened on #6650: a test built a fake session token out of a real JWT header segment. The value was inert and never a live credential, but it read as one to the scanner, and removing it in a later commit did not clear the check.
What it says
The existing convention, which seven test files already follow:
And the rule that was missing: never build a fixture out of a real credential's structure. When a test needs a value with internal structure, keep the structure it actually exercises and make everything else unmistakably fake.
The paragraph describes those shapes rather than quoting one, since a document warning about credential-shaped literals is a poor place to leave one.
Test plan