-
Notifications
You must be signed in to change notification settings - Fork 0
feat: npm provenance + publish-token health actions #3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
0b53d49
c6106ee
26ed4ca
ea42d3f
d46e941
0b0ecea
9971acf
6dbd82b
d60ab38
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| name: CI | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main] | ||
| pull_request: | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| test: | ||
| name: Test composite actions | ||
| runs-on: ubuntu-24.04 | ||
| timeout-minutes: 10 | ||
| steps: | ||
| - name: Harden Runner | ||
| uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0 | ||
| with: | ||
| egress-policy: audit | ||
| - name: Checkout | ||
| uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 | ||
| with: | ||
| persist-credentials: false | ||
| - name: Run action tests | ||
| # A bare directory arg worked on Node 20 but fails on Node 24 | ||
| # (MODULE_NOT_FOUND); runners now default to 24. Use an explicit glob. | ||
| run: node --test "actions/**/test/*.test.js" |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,79 @@ | ||
| name: Probe publish token | ||
| description: >- | ||
| Liveness probe for a marketplace publish token using the vendor CLI's own | ||
| verify-pat command. The token is passed only via environment — never argv — | ||
| and neither the token nor any response body is logged. | ||
|
|
||
| inputs: | ||
| tool: | ||
| description: "'vsce' (VS Code Marketplace) or 'ovsx' (Open VSX)" | ||
| required: true | ||
| namespace: | ||
| description: "Publisher / namespace to verify against, e.g. nimbus-agent" | ||
| required: true | ||
|
|
||
| outputs: | ||
| status: | ||
| description: "ok | dead | not-configured | indeterminate" | ||
| value: ${{ steps.probe.outputs.status }} | ||
|
|
||
| runs: | ||
| using: composite | ||
| steps: | ||
| - id: probe | ||
| shell: bash | ||
| # PUBLISH_TOKEN is mapped by the caller from the appropriate secret. | ||
| # It is exported into the vendor CLI's expected env var and never echoed. | ||
| # Output is discarded wholesale: error messages from these CLIs can echo | ||
| # request context, and this runs in a public repository's logs. | ||
| # | ||
| # CLI versions are pinned EXACTLY, not by range: this hands a live publish | ||
| # credential to the resolved package, so `@latest` (or a `^` range, which | ||
| # still floats) would put a third-party release in the credential's trust | ||
| # boundary. Bumping these is a deliberate, reviewed change. | ||
| run: | | ||
| set +e | ||
| if [ -z "${PUBLISH_TOKEN}" ]; then | ||
| # An unset secret cannot be assessed as live or revoked: this branch | ||
| # never contacts the vendor CLI and never runs the reachability | ||
| # check below, so it has no evidence the token was rejected. | ||
| # `dead` is reserved for a confirmed rejection by a reachable | ||
| # service — reporting it here would be a false revocation alarm. | ||
| # Whether an unconfigured token is a hard failure is the caller's | ||
| # call; this probe's job is only to report the signal honestly. | ||
| echo "status=not-configured" >> "$GITHUB_OUTPUT" | ||
| echo "probe ${TOOL}: secret not configured (no token to check)" | ||
| exit 0 | ||
| fi | ||
| if [ "${TOOL}" = "vsce" ]; then | ||
| VSCE_PAT="${PUBLISH_TOKEN}" npx --yes @vscode/vsce@3.9.2 verify-pat "${NAMESPACE}" >/dev/null 2>&1 | ||
| code=$? | ||
| reach_url="https://marketplace.visualstudio.com/" | ||
| else | ||
| OVSX_PAT="${PUBLISH_TOKEN}" npx --yes ovsx@1.0.2 verify-pat "${NAMESPACE}" >/dev/null 2>&1 | ||
| code=$? | ||
| reach_url="https://open-vsx.org/api/-/search?size=1" | ||
| fi | ||
|
|
||
| if [ $code -eq 0 ]; then | ||
| echo "status=ok" >> "$GITHUB_OUTPUT" | ||
| echo "probe ${TOOL} (${NAMESPACE}): ok" | ||
| exit 0 | ||
| fi | ||
|
|
||
| # A non-zero exit alone does NOT mean the token is revoked — registry | ||
| # downtime, a timeout or rate-limiting look identical. Only call it dead | ||
| # if the service is actually reachable; otherwise report indeterminate | ||
| # so the caller warns instead of raising a false revocation alarm. | ||
| http="$(curl -sS -o /dev/null -w '%{http_code}' --max-time 20 "$reach_url" 2>/dev/null)" | ||
| if [ "$http" = "200" ]; then | ||
| echo "status=dead" >> "$GITHUB_OUTPUT" | ||
| echo "probe ${TOOL} (${NAMESPACE}): failed (exit=${code}) and service reachable -> dead" | ||
| else | ||
| echo "status=indeterminate" >> "$GITHUB_OUTPUT" | ||
| echo "probe ${TOOL} (${NAMESPACE}): failed (exit=${code}) and service unreachable (http=${http}) -> indeterminate" | ||
|
Comment on lines
+64
to
+74
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🌐 Web query:
💡 Result: For the Citations:
🏁 Script executed: #!/bin/bash
set -euo pipefail
# Map the action file, then inspect the relevant range with line numbers.
wc -l actions/probe-publish-token/action.yml
sed -n '1,220p' actions/probe-publish-token/action.yml | cat -nRepository: nimbus-agent/.github Length of output: 4286 Don’t map every non-zero 🤖 Prompt for AI Agents |
||
| fi | ||
| exit 0 | ||
| env: | ||
| TOOL: ${{ inputs.tool }} | ||
| NAMESPACE: ${{ inputs.namespace }} | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| name: Verify npm provenance | ||
| description: >- | ||
| Assert a published npm version carries both the npm publish attestation and a | ||
| SLSA provenance predicate, and that the provenance names the expected source | ||
| repo/workflow/commit. Checks claims, not cryptography — pair with | ||
| `npm audit signatures` for signature verification. | ||
|
|
||
| inputs: | ||
| package: | ||
| description: "npm package name, e.g. @nimbus-dev/sdk" | ||
| required: true | ||
| version: | ||
| description: "Published version to verify" | ||
| required: true | ||
| expected-repo: | ||
| description: "owner/repo that must appear in the provenance source claim" | ||
| required: true | ||
| expected-workflow: | ||
| description: "Workflow path that must have produced it, e.g. .github/workflows/release.yml" | ||
| required: false | ||
| expected-sha: | ||
| description: "Commit SHA the build must have come from" | ||
| required: false | ||
| severity: | ||
| description: "'gate' (default) fails the job on any non-ok status; 'monitor' always exits 0" | ||
| required: false | ||
| default: gate | ||
|
|
||
| outputs: | ||
| status: | ||
| description: "ok | missing-provenance | source-mismatch | indeterminate" | ||
| value: ${{ steps.verify.outputs.status }} | ||
| detail: | ||
| description: "Short human-readable reason" | ||
| value: ${{ steps.verify.outputs.detail }} | ||
|
|
||
| runs: | ||
| using: composite | ||
| steps: | ||
| - id: verify | ||
| shell: bash | ||
| # Node 20+ is preinstalled on GitHub runners; no setup-node, no install. | ||
| run: node "${{ github.action_path }}/src/main.js" | ||
| env: | ||
| INPUT_PACKAGE: ${{ inputs.package }} | ||
| INPUT_VERSION: ${{ inputs.version }} | ||
| INPUT_EXPECTED_REPO: ${{ inputs.expected-repo }} | ||
| INPUT_EXPECTED_WORKFLOW: ${{ inputs.expected-workflow }} | ||
| INPUT_EXPECTED_SHA: ${{ inputs.expected-sha }} | ||
| INPUT_SEVERITY: ${{ inputs.severity }} |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| export const PUBLISH_PREDICATE = "https://github.com/npm/attestation/tree/main/specs/publish/v0.1"; | ||
| export const SLSA_PREDICATE = "https://slsa.dev/provenance/v1"; | ||
|
|
||
| /** | ||
| * Decode the DSSE payloads out of a registry attestation response. | ||
| * | ||
| * The attestation content lives in `bundle.dsseEnvelope.payload` (base64 JSON), | ||
| * NOT at the top level. We read predicateType from the decoded statement rather | ||
| * than the unsigned outer wrapper. | ||
| * | ||
| * Returns null on any unrecognised shape — the caller maps that to | ||
| * "indeterminate", never a false "ok". | ||
| */ | ||
| export function decodeStatements(body) { | ||
| if (body === null || typeof body !== "object") return null; | ||
| const list = body.attestations; | ||
| if (!Array.isArray(list)) return null; | ||
| const statements = []; | ||
| for (const att of list) { | ||
| const payload = att?.bundle?.dsseEnvelope?.payload; | ||
| if (typeof payload !== "string") return null; | ||
| let decoded; | ||
| try { | ||
| decoded = JSON.parse(Buffer.from(payload, "base64").toString("utf8")); | ||
| } catch { | ||
| return null; | ||
| } | ||
| if (decoded === null || typeof decoded !== "object") return null; | ||
| statements.push(decoded); | ||
| } | ||
| return statements; | ||
| } | ||
|
|
||
| function fail(status, detail) { | ||
| return { status, detail }; | ||
| } | ||
|
|
||
| /** | ||
| * Assert the attestation set is complete AND attested to the expected source. | ||
| * "An attestation exists" is a weaker claim than "attested to us". | ||
| */ | ||
| export function classifyProvenance(statements, expected) { | ||
| if (statements === null) return fail("indeterminate", "unparseable attestation response"); | ||
| if (statements.length === 0) return fail("missing-provenance", "no attestations published"); | ||
|
|
||
| const types = new Set(statements.map((s) => s?.predicateType)); | ||
| if (!types.has(PUBLISH_PREDICATE)) { | ||
| return fail("missing-provenance", "no npm publish attestation"); | ||
| } | ||
| const slsa = statements.find((s) => s?.predicateType === SLSA_PREDICATE); | ||
| if (slsa === undefined) { | ||
| return fail("missing-provenance", "no SLSA provenance predicate — publish degraded"); | ||
| } | ||
|
|
||
| const wf = slsa?.predicate?.buildDefinition?.externalParameters?.workflow; | ||
| if (wf === null || typeof wf !== "object") { | ||
| return fail("indeterminate", "provenance carries no workflow claim"); | ||
| } | ||
|
|
||
| const wantRepo = `https://github.com/${expected.repo}`; | ||
| if (wf.repository !== wantRepo) { | ||
| return fail("source-mismatch", `repository ${String(wf.repository)} != ${wantRepo}`); | ||
| } | ||
| if (expected.workflow !== undefined && wf.path !== expected.workflow) { | ||
| return fail("source-mismatch", `workflow ${String(wf.path)} != ${expected.workflow}`); | ||
| } | ||
| if (expected.sha !== undefined) { | ||
| const deps = slsa?.predicate?.buildDefinition?.resolvedDependencies; | ||
| const commit = Array.isArray(deps) | ||
| ? deps.find((d) => typeof d?.digest?.gitCommit === "string")?.digest?.gitCommit | ||
| : undefined; | ||
| if (commit !== expected.sha) { | ||
| return fail("source-mismatch", `commit ${String(commit)} != ${expected.sha}`); | ||
| } | ||
| } | ||
| return { status: "ok", detail: `${String(wf.path)} @ ${String(wf.repository)}` }; | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| /** | ||
| * Deterministic retry schedule: 5s doubling to a 30s cap, ~2.5 min total. | ||
| * | ||
| * Jitter is deliberately omitted. Jitter decorrelates a FLEET of clients | ||
| * retrying in lockstep; here exactly one client retries per publish, so it | ||
| * would only add nondeterminism to these tests. Do not re-add it. | ||
| */ | ||
| export const BACKOFF_MS = Object.freeze([5000, 10000, 20000, 30000, 30000, 30000, 30000]); | ||
|
|
||
| const REGISTRY = "https://registry.npmjs.org/-/npm/v1/attestations"; | ||
|
|
||
| /** | ||
| * All three encodings of a scoped name (raw, fully percent-encoded, mixed) | ||
| * return HTTP 200 — verified against the live registry 2026-07-19. The raw | ||
| * form is canonical here; no encoding is required. | ||
| */ | ||
| export function attestationUrl(pkg, version) { | ||
| return `${REGISTRY}/${pkg}@${version}`; | ||
| } | ||
|
|
||
| /** | ||
| * Attestations can trail a publish, and the registry is CDN-fronted, so a 404 | ||
| * is only conclusive once the backoff schedule is exhausted. | ||
| * | ||
| * Distinguishes "absent" (404 throughout — the package genuinely has no | ||
| * attestation) from "error" (5xx / network / unparseable). Callers map those | ||
| * to different severities. | ||
| */ | ||
| export async function fetchAttestations(pkg, version, deps) { | ||
| const backoff = deps.backoff ?? BACKOFF_MS; | ||
| const url = attestationUrl(pkg, version); | ||
| let lastDetail = "no attempt made"; | ||
|
|
||
| for (let attempt = 0; attempt <= backoff.length; attempt += 1) { | ||
| if (attempt > 0) await deps.sleep(backoff[attempt - 1]); | ||
| try { | ||
| const res = await deps.fetchFn(url, { headers: { accept: "application/json" } }); | ||
| if (res.status === 200) { | ||
| try { | ||
| return { outcome: "body", body: await res.json(), detail: "200" }; | ||
| } catch { | ||
| // A 200 carrying unparseable bytes is transient, not authoritative: | ||
| // a CDN/proxy error page or a truncated body both look like this. | ||
| // Retry rather than failing a release on one bad edge response. | ||
| lastDetail = "200 with unparseable JSON body"; | ||
| continue; | ||
| } | ||
| } | ||
| if (res.status === 404) { | ||
| lastDetail = "404 after full backoff"; | ||
| continue; | ||
| } | ||
| lastDetail = `HTTP ${res.status}`; | ||
| } catch { | ||
| // Never surface the thrown error object: fetch errors can embed request | ||
| // headers, and this action runs in public logs. | ||
| lastDetail = "network error"; | ||
| } | ||
| } | ||
| return lastDetail === "404 after full backoff" | ||
| ? { outcome: "absent", detail: lastDetail } | ||
| : { outcome: "error", detail: lastDetail }; | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| import { appendFileSync } from "node:fs"; | ||
| import { classifyProvenance, decodeStatements } from "./classify.js"; | ||
| import { fetchAttestations } from "./fetch-attestations.js"; | ||
|
|
||
| /** Map a fetch outcome plus the classifier into a single status. */ | ||
| export function decide(fetched, expected) { | ||
| if (fetched.outcome === "absent") { | ||
| return { status: "missing-provenance", detail: fetched.detail }; | ||
| } | ||
| if (fetched.outcome === "error") { | ||
| // A registry problem is NOT evidence the publish was bad. | ||
| return { status: "indeterminate", detail: fetched.detail }; | ||
| } | ||
| return classifyProvenance(decodeStatements(fetched.body), expected); | ||
| } | ||
|
|
||
| /** | ||
| * The gate must not let a possibly-degraded publish through; the monitor must | ||
| * not turn a registry hiccup into issue spam. Same status, different severity. | ||
| */ | ||
| export function exitCodeFor(status, severity) { | ||
| if (severity === "monitor") return 0; | ||
| return status === "ok" ? 0 : 1; | ||
| } | ||
|
|
||
| function runbook(pkg, version, status, detail) { | ||
| return [ | ||
| "::error::npm provenance verification FAILED", | ||
| `::error::package=${pkg}@${version} status=${status} detail=${detail}`, | ||
| "::error::RUNBOOK:", | ||
| "::error:: 1. This version is already on the registry. npm allows unpublish", | ||
| "::error:: only within 72h of publish — check the publish timestamp now.", | ||
| "::error:: 2. Within 72h: `npm unpublish <pkg>@<version>`, fix the cause, republish.", | ||
| '::error:: 3. After 72h: `npm deprecate <pkg>@<version> "no provenance; use <next>"`', | ||
| "::error:: then publish a patch version with provenance.", | ||
| "::error:: 4. Common causes: `id-token: write` missing from the publish job;", | ||
| "::error:: npm older than 11.5.1; trusted-publisher binding removed on npmjs.com.", | ||
| ].join("\n"); | ||
| } | ||
|
Comment on lines
+26
to
+39
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔒 Security & Privacy | 🔴 Critical | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: set -euo pipefail
printf '--- files ---\n'
git ls-files actions/verify-npm-provenance/src/main.js actions/verify-npm-provenance/src/classify.js actions/verify-npm-provenance/.github/workflows 2>/dev/null || true
printf '\n--- main.js outline ---\n'
ast-grep outline actions/verify-npm-provenance/src/main.js --view expanded || true
printf '\n--- classify.js outline ---\n'
ast-grep outline actions/verify-npm-provenance/src/classify.js --view expanded || true
printf '\n--- relevant lines main.js ---\n'
cat -n actions/verify-npm-provenance/src/main.js | sed -n '1,220p'
printf '\n--- relevant lines classify.js ---\n'
cat -n actions/verify-npm-provenance/src/classify.js | sed -n '1,220p'
printf '\n--- search for GITHUB_OUTPUT usage ---\n'
rg -n "GITHUB_OUTPUT|appendFileSync\\(|console\\.log\\(|runbook\\(" actions/verify-npm-provenance/srcRepository: nimbus-agent/.github Length of output: 249 🏁 Script executed: set -euo pipefail
python3 - <<'PY'
from pathlib import Path
p = Path('actions/verify-npm-provenance/src/main.js')
print(p.exists(), p.stat().st_size if p.exists() else None)
PYRepository: nimbus-agent/.github Length of output: 168 🏁 Script executed: set -euo pipefail
echo '--- main.js outline ---'
ast-grep outline actions/verify-npm-provenance/src/main.js --view expanded || true
echo
echo '--- classify.js outline ---'
ast-grep outline actions/verify-npm-provenance/src/classify.js --view expanded || true
echo
echo '--- main.js lines ---'
cat -n actions/verify-npm-provenance/src/main.js | sed -n '1,220p'
echo
echo '--- classify.js lines ---'
cat -n actions/verify-npm-provenance/src/classify.js | sed -n '1,220p'
echo
echo '--- output consumption search ---'
rg -n "steps\\.verify\\.outputs\\.status|outputs\\.status|GITHUB_OUTPUT|appendFileSync\\(|console\\.log\\(|runbook\\(" actions/verify-npm-provenance/src .github actions -g '!**/node_modules/**'Repository: nimbus-agent/.github Length of output: 10093 🌐 Web query:
💡 Result: To set a multiline output in GitHub Actions using the GITHUB_OUTPUT environment file, you should use a heredoc-style delimiter [1][2]. The official and recommended approach is to append the output name followed by a unique delimiter, the multiline value, and the closing delimiter to the file specified by the $GITHUB_OUTPUT environment variable [1][3][4]. The standard syntax is as follows: {name}<<{delimiter} {value} {delimiter} For example, using bash: { echo 'MY_OUTPUT<<EOF' echo 'line 1' echo 'line 2' echo 'EOF' } >> "$GITHUB_OUTPUT" Important Considerations: 1. Delimiter Uniqueness: Ensure the chosen delimiter (e.g., EOF) does not appear within your actual value [1][5][6]. If the value contains arbitrary text that might include the delimiter, this method is unsafe, and you should instead write the output to a temporary file [1]. 2. Closing Delimiter Requirements: The closing delimiter must be on its own line [1][4]. A common point of failure is when the value being output does not end with a newline character, causing the closing delimiter to be appended immediately after the content [4]. To ensure reliability, especially when cat-ing files or command outputs, explicitly add a newline before the closing delimiter, for example by using printf '\n' [4]. 3. Escaping: Unlike older methods (like the deprecated::set-output command which required manual escaping of characters like %0A or %0D), the GITHUB_OUTPUT file mechanism natively handles multiline strings through the delimiter syntax [3][7]. You do not need to manually escape newline or carriage return characters when using the heredoc approach [4][7]. 4. Actions/Toolkit: The Citations:
🌐 Web query:
💡 Result: Writing to the GITHUB_OUTPUT environment file is a common method for passing data between steps, but it introduces significant security and parsing risks if not handled correctly [1][2]. ### Core Mechanisms and Risks When you append content to the GITHUB_OUTPUT file, the runner parses the file to identify new outputs [3][4]. The primary risks involve: 1. Newline Injection: If untrusted input containing newlines is written to GITHUB_OUTPUT using the simple Citations:
Untrusted provenance details can clobber step outputs
🤖 Prompt for AI Agents |
||
|
|
||
| if (process.env["NODE_ENV"] !== "test" && process.argv[1]?.endsWith("main.js")) { | ||
| const pkg = process.env["INPUT_PACKAGE"] ?? ""; | ||
| const version = process.env["INPUT_VERSION"] ?? ""; | ||
| const severity = process.env["INPUT_SEVERITY"] === "monitor" ? "monitor" : "gate"; | ||
| const expected = { repo: process.env["INPUT_EXPECTED_REPO"] ?? "" }; | ||
| const wf = process.env["INPUT_EXPECTED_WORKFLOW"]; | ||
| const sha = process.env["INPUT_EXPECTED_SHA"]; | ||
| if (wf) expected.workflow = wf; | ||
| if (sha) expected.sha = sha; | ||
|
|
||
| const fetched = await fetchAttestations(pkg, version, { | ||
| fetchFn: fetch, | ||
| sleep: (ms) => new Promise((r) => setTimeout(r, ms)), | ||
| }); | ||
| const { status, detail } = decide(fetched, expected); | ||
|
|
||
| const out = process.env["GITHUB_OUTPUT"]; | ||
| if (out) appendFileSync(out, `status=${status}\ndetail=${detail}\n`); | ||
| console.log(`npm provenance: ${pkg}@${version} -> ${status} (${detail})`); | ||
| if (status !== "ok" && severity === "gate") console.log(runbook(pkg, version, status, detail)); | ||
| // Set exitCode instead of calling process.exit(): exit() tears the process | ||
| // down immediately, which can race undici's keep-alive sockets/timers | ||
| // (observed as a libuv UV_HANDLE_CLOSING assertion crash on Windows/Node 24) | ||
| // and can truncate buffered stdout / the GITHUB_OUTPUT write. Assigning | ||
| // process.exitCode lets Node exit naturally once the event loop drains, | ||
| // preserving the same exit-code contract without the early teardown. | ||
| process.exitCode = exitCodeFor(status, severity); | ||
| } | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
Reject unsupported
toolvalues before reading the token.Any typo other than
vscefalls into the Open VSX branch, potentially sending a VSCE token to Open VSX. Validatevsce|ovsxbefore the token-presence check and returnindeterminatefor unsupported values.Proposed fix
📝 Committable suggestion
🤖 Prompt for AI Agents