Audit release environment settings

.github/workflows/release.yml

@@ -37,10 +37,12 @@
 #
 # Provenance / attestation, soup to nuts (all genuine, nothing faked):
 #   * SLSA generator -> `*.intoto.jsonl` on the Release  (OpenSSF Scorecard
-#     Signed-Releases provenance probe -> 10/10; SLSA Build L3).
+#     Signed-Releases provenance probe; older unsigned releases may keep that
+#     score below 10 temporarily; SLSA Build L3).
 #   * actions/attest-build-provenance -> GitHub attestation store + a
 #     `*.sigstore.json` bundle on the Release (`gh attestation verify`; also the
-#     Scorecard signing probe -> 8, a backup if the .intoto.jsonl ever regresses).
+#     Scorecard signing probe sees this asset as a backup if the .intoto.jsonl
+#     ever regresses).
 #   * gh-action-pypi-publish -> PEP 740 attestations on PyPI (Integrity API).
 #   * post-publish PyPI JSON hash check -> every served wheel/sdist digest
 #     matches the staged dist files.
@@ -218,8 +220,9 @@ jobs:
         with:
           toolchain: stable
       - name: Package the crate
-        # Emits target/package/ordvec-<version>.crate — the same content
-        # `cargo publish` uploads, so the provenance covers the published artifact.
+        # Emits the SLSA-attested .crate artifact. `publish-crate` later
+        # compares both a local repackage and the crates.io-served artifact to
+        # this file.
         run: cargo package -p ordvec --locked
       - name: Generate CycloneDX SBOM for the crate
         run: |

Cargo.toml

@@ -42,6 +42,7 @@ exclude = [
     "ordvec-manifest/",
     "ordvec-python/",
     "tests/__pycache__/",
+    "tests/release_environment_settings.sh",
     "tests/release_publish_invariants.py",
     "tests/release_publish_invariants.sh",
     "tests/release_signed_release_invariants.sh",

README.md

@@ -263,13 +263,14 @@ Collaboration we're actively seeking:
 - **Independent reproduction** — re-running the benchmark on other hardware
   and reporting the numbers.
 
-If that's your area, see [GOVERNANCE.md](GOVERNANCE.md) and open an issue or a
-discussion.
+If that's your area, see
+[GOVERNANCE.md](https://github.com/Fieldnote-Echo/ordvec/blob/main/GOVERNANCE.md)
+and open an issue or a discussion.
 
 ## Contributing
 
 Contributions to the code, the docs, and the paper are all welcome — see
-[CONTRIBUTING.md](CONTRIBUTING.md).
+[CONTRIBUTING.md](https://github.com/Fieldnote-Echo/ordvec/blob/main/CONTRIBUTING.md).
 
 ## Minimum supported Rust version
 

RELEASING.md

@@ -126,8 +126,18 @@ filename. Until either is updated, the corresponding gated publish fails
      strategy. An interior commit that exists in history only from a PR branch
      has no push-to-main run (its CI ran as a `pull_request` on the branch)
      and so is not releasable.
-4. Get the maintainer's explicit go to publish.
-5. Push the version tag from `main` (signed):
+4. Run the manual release-settings audit before creating the tag:
+
+   ```sh
+   bash tests/release_environment_settings.sh
+   ```
+
+   This verifies the GitHub Environments still require the expected reviewer
+   and accept only the stable release tag pattern. Separately verify the
+   registry Trusted Publisher records by hand: crates.io must point to
+   `release.yml` / `crates-io`, and PyPI must point to `release.yml` / `pypi`.
+5. Get the maintainer's explicit go to publish.
+6. Push the version tag from `main` (signed):
 
    ```sh
    git tag -s vX.Y.Z -m "vX.Y.Z"
@@ -139,7 +149,7 @@ filename. Until either is updated, the corresponding gated publish fails
    generates the SLSA `*.intoto.jsonl`; and stages every artifact, the
    attestation bundle, and the provenance on the GitHub Release — **as a
    DRAFT**. It then pauses at the two registry environment gates.
-6. **Approve the two publish environments** when they pause in the Actions UI
+7. **Approve the two publish environments** when they pause in the Actions UI
    (one for `crates-io`, one for `pypi`). The required-reviewer approval is
    what authorises the registry push.
    - `publish-crate` first sha256-compares its repackaged `.crate` to the
@@ -152,7 +162,7 @@ filename. Until either is updated, the corresponding gated publish fails
    - `publish-pypi` also queries PyPI after upload and compares every served
      wheel/sdist SHA-256 digest against the staged `dist/` files before the
      GitHub Release can un-draft.
-7. Verify each published artifact and its provenance:
+8. Verify each published artifact and its provenance:
    - crates.io / docs.rs;
    - PyPI (confirm the post-publish hash-verification log, optionally
      `pip download ordvec==X.Y.Z` and inspect, plus check the PEP 740 attestation

tests/release_environment_settings.sh

@@ -0,0 +1,107 @@
+#!/usr/bin/env bash
+#
+# Manual pre-tag audit for GitHub Environment release gates.
+#
+# This is intentionally not a normal CI check: it requires an authenticated
+# gh token that can read repository environment settings.
+set -euo pipefail
+
+REPO="${REPO:-Fieldnote-Echo/ordvec}"
+EXPECTED_REVIEWER="${EXPECTED_REVIEWER:-Fieldnote-Echo}"
+EXPECTED_POLICY="${EXPECTED_POLICY:-v[0-9]*.[0-9]*.[0-9]*}"
+ENVIRONMENTS=(crates-io pypi)
+
+fail() {
+  echo "::error::release environment settings audit failed: $*"
+  exit 1
+}
+
+api_jq() {
+  local path="$1"
+  local filter="$2"
+  local err output stderr
+
+  if ! err="$(mktemp)"; then
+    fail "could not create temporary file for gh api stderr"
+  fi
+
+  if ! output="$(gh api "$path" --jq "$filter" 2>"$err")"; then
+    stderr="$(cat "$err")"
+    rm -f "$err"
+    fail "cannot read ${path}; authenticate with a token that can read ${REPO} repository environment settings. gh api: ${stderr}"
+  fi
+  rm -f "$err"
+
+  printf '%s\n' "$output"
+}
+
+command -v gh >/dev/null 2>&1 \
+  || fail "gh CLI not found; install GitHub CLI (gh) and authenticate before running this audit"
+
+if ! gh auth status -h github.com; then
+  fail "gh auth status failed; run gh auth login with an account/token that can read ${REPO} repository environment settings"
+fi
+
+check_environment() {
+  local env="$1"
+  local env_path="repos/${REPO}/environments/${env}"
+  local policies_path="${env_path}/deployment-branch-policies?per_page=100"
+  local env_data policies_data
+  local env_name required_rule_count reviewer_count reviewer_summary
+  local custom_branch_policies protected_branches
+  local policy_total policy_summary policy_type policy_name
+
+  echo "Auditing ${REPO} environment ${env}..."
+
+  env_data="$(api_jq "$env_path" '[
+    (.name // ""),
+    ([.protection_rules[]? | select(.type == "required_reviewers")] | length | tostring),
+    ([.protection_rules[]? | select(.type == "required_reviewers") | .reviewers[]?] | length | tostring),
+    ([.protection_rules[]? | select(.type == "required_reviewers") | .reviewers[]? | "\(.type):\(.reviewer.login // .reviewer.slug // .reviewer.name // "unknown")"] | join(", ")),
+    (.deployment_branch_policy.custom_branch_policies | tostring),
+    (.deployment_branch_policy.protected_branches | tostring)
+  ] | @tsv')"
+  IFS=$'\t' read -r env_name required_rule_count reviewer_count reviewer_summary custom_branch_policies protected_branches <<< "$env_data"
+
+  [ "$env_name" = "$env" ] \
+    || fail "${env}: environment not found"
+
+  [ "$required_rule_count" = "1" ] \
+    || fail "${env}: expected exactly one required_reviewers protection rule; found ${required_rule_count}"
+
+  [ "$reviewer_count" = "1" ] \
+    || fail "${env}: expected exactly one required reviewer User:${EXPECTED_REVIEWER}; found ${reviewer_count} (${reviewer_summary:-none})"
+  [ "$reviewer_summary" = "User:${EXPECTED_REVIEWER}" ] \
+    || fail "${env}: expected required reviewer User:${EXPECTED_REVIEWER}; found ${reviewer_summary:-none}"
+
+  [ "$custom_branch_policies" = "true" ] \
+    || fail "${env}: expected deployment_branch_policy.custom_branch_policies == true; found ${custom_branch_policies}"
+
+  [ "$protected_branches" = "false" ] \
+    || fail "${env}: expected deployment_branch_policy.protected_branches == false; found ${protected_branches}"
+
+  policies_data="$(api_jq "$policies_path" '[
+    (.total_count | tostring),
+    ([.branch_policies[]? | "\(.type):\(.name)"] | join(", ")),
+    (.branch_policies[0].type // ""),
+    (.branch_policies[0].name // "")
+  ] | @tsv')"
+  IFS=$'\t' read -r policy_total policy_summary policy_type policy_name <<< "$policies_data"
+
+  [ "$policy_total" = "1" ] \
+    || fail "${env}: expected exactly one deployment branch/tag policy tag:${EXPECTED_POLICY}; found ${policy_total} (${policy_summary:-none})"
+
+  [ "$policy_type" = "tag" ] \
+    || fail "${env}: expected deployment policy type tag; found ${policy_type:-none}"
+
+  [ "$policy_name" = "$EXPECTED_POLICY" ] \
+    || fail "${env}: expected deployment policy name ${EXPECTED_POLICY}; found ${policy_name:-none}"
+
+  echo "OK: ${env} requires User:${EXPECTED_REVIEWER} and only tag:${EXPECTED_POLICY}."
+}
+
+for env in "${ENVIRONMENTS[@]}"; do
+  check_environment "$env"
+done
+
+echo "OK: release environment settings match the pre-tag policy."

tests/release_signed_release_invariants.sh

@@ -2,8 +2,10 @@
 #
 # Signed-release / provenance invariants — pinned in CI.
 #
-# release.yml's signed-release graph is what gets us OpenSSF Scorecard
-# Signed-Releases = 10 and keeps the build-attest-publish chain honest:
+# release.yml's signed-release graph attaches the .intoto.jsonl and Sigstore
+# assets that OpenSSF Scorecard detects for Signed-Releases, while older
+# unsigned releases may keep the score below 10 temporarily. The same graph
+# keeps the build-attest-publish chain honest:
 #
 #     build-{crate,wheels,sdist}                  (artifacts)
 #         |