Skip to content

Public-release hardening: drop neocache, reproducibility & CI fixes#7

Merged
sam-obeid merged 1 commit intomainfrom
fix/gate-neocache-behind-feature
May 4, 2026
Merged

Public-release hardening: drop neocache, reproducibility & CI fixes#7
sam-obeid merged 1 commit intomainfrom
fix/gate-neocache-behind-feature

Conversation

@sam-obeid
Copy link
Copy Markdown
Collaborator

Single squashed commit covering everything needed to make a fresh public clone build, lint, test and release-build green from a CI runner with no Shopify git credentials.

Drop the neocache benchmark from the public repo

  • Remove the Shopify/neocache git dependency from Cargo.toml and its package stanza + cache-bench dep entry from Cargo.lock.
  • Delete src/caches/neocache.rs.
  • Remove neocache from ALL_CACHES and the dispatch arm in src/main.rs, and from the AVAILABLE CACHES / Available caches blocks in --help and --info output.
  • Replace the [neocache] row in the README comparison table with a 'Note on neocache' explainer documenting why it is absent and how to re-add it on a private overlay branch.

Rationale: cargo always fetches git sources referenced by
Cargo.lock, even for optional = true deps with the feature
disabled, so feature-gating cannot fix the auth failure on public
CI runners. Verified empirically: cargo clippy --all-targets --locked -- -D warnings now passes from a fresh CARGO_HOME with
no git creds (the exact public-CI scenario), where it previously
failed with 'failed to authenticate when downloading repository'.

CI workflow + supply-chain hardening

  • .github/workflows/ci.yml: pin all third-party actions to 40-char SHAs (with version comment for Dependabot), lock GITHUB_TOKEN to contents: read, build the cargo cache key from OS+toolchain+ Cargo.lock with a toolchain-scoped restore-key, fail-fast off, and add a cargo audit --deny warnings job with a cached cargo-audit binary.
  • .cargo/audit.toml: explicitly-accepted advisories with rationale and review dates; new advisories fail the build by default.
  • .github/CODEOWNERS: ownership for review routing.
  • Cargo.toml: rust-version = "1.94" MSRV, publish = false (the cache-bench name is unclaimed on crates.io and we have no intent to claim it), readme/keywords/categories metadata.

Reproducibility plumbing

  • generate_value_pool now takes the base RNG seed and derives a deterministic StdRng from it (with a 0xC0FFEE offset to avoid colliding with per-task seeds), so re-running with the same --seed produces bit-identical value bytes.
  • generate_fixed_value takes the RNG by reference instead of using an unseeded thread RNG.
  • calculate_statistics: f64::total_cmp instead of partial_cmp so a stray NaN sorts deterministically rather than panicking.
  • BenchConfig::cold_start docs spell out that it is forced true during the calibration pass.
  • Document the fresh-key window wraparound for write_new_key_fraction > 0 in total_benchmark_keys.

CLI polish

  • ANSI colour escapes are now emitted only when stdout is a TTY (cached behind OnceLock); piping benchmark output to a file or CI log no longer produces literal \x1b[...m sequences.

Docs

  • README: refreshed benchmark-output screenshot URL (kept the new user-attachments image from Update README.md #6 over the in-repo placeholder that referenced a not-yet-committed file).
  • CONTRIBUTING.md: clarifications.
  • docs/assets/README.md: contributor guidance scaffolding.

Verification (local, all green):

  • cargo fmt --check
  • cargo clippy --all-targets --locked -- -D warnings (also from a fresh empty CARGO_HOME with no git creds)
  • cargo build --locked
  • cargo build --release --locked
  • cargo test --locked (20 passed)

What does this PR do?

Why

Checklist

  • cargo fmt --check passes
  • cargo clippy --all-targets --locked -- -D warnings passes
  • cargo test --locked passes
  • cargo build --release --locked passes
  • If this changes benchmark output: I have included before / after
    numbers from the same machine in this PR description
  • If this adds a new cache: I have updated ALL_CACHES and
    dispatch() in src/main.rs and the README's caches table

Benchmark output (if applicable)

before
paste the `cargo run --release` table here
after
paste the `cargo run --release` table here

Single squashed commit covering everything needed to make a fresh
public clone build, lint, test and release-build green from a CI
runner with no Shopify git credentials.

Drop the neocache benchmark from the public repo
  - Remove the `Shopify/neocache` git dependency from Cargo.toml and
    its package stanza + cache-bench dep entry from Cargo.lock.
  - Delete src/caches/neocache.rs.
  - Remove neocache from ALL_CACHES and the dispatch arm in
    src/main.rs, and from the AVAILABLE CACHES / Available caches
    blocks in --help and --info output.
  - Replace the [neocache] row in the README comparison table with
    a 'Note on neocache' explainer documenting why it is absent and
    how to re-add it on a private overlay branch.

  Rationale: cargo always fetches git sources referenced by
  Cargo.lock, even for `optional = true` deps with the feature
  disabled, so feature-gating cannot fix the auth failure on public
  CI runners. Verified empirically: `cargo clippy --all-targets
  --locked -- -D warnings` now passes from a fresh CARGO_HOME with
  no git creds (the exact public-CI scenario), where it previously
  failed with 'failed to authenticate when downloading repository'.

CI workflow + supply-chain hardening
  - .github/workflows/ci.yml: pin all third-party actions to 40-char
    SHAs (with version comment for Dependabot), lock GITHUB_TOKEN to
    `contents: read`, build the cargo cache key from OS+toolchain+
    Cargo.lock with a toolchain-scoped restore-key, fail-fast off,
    and add a `cargo audit --deny warnings` job with a cached
    cargo-audit binary.
  - .cargo/audit.toml: explicitly-accepted advisories with rationale
    and review dates; new advisories fail the build by default.
  - .github/CODEOWNERS: ownership for review routing.
  - Cargo.toml: `rust-version = "1.94"` MSRV, `publish = false`
    (the `cache-bench` name is unclaimed on crates.io and we have
    no intent to claim it), readme/keywords/categories metadata.

Reproducibility plumbing
  - generate_value_pool now takes the base RNG seed and derives a
    deterministic StdRng from it (with a 0xC0FFEE offset to avoid
    colliding with per-task seeds), so re-running with the same
    --seed produces bit-identical value bytes.
  - generate_fixed_value takes the RNG by reference instead of using
    an unseeded thread RNG.
  - calculate_statistics: f64::total_cmp instead of partial_cmp so a
    stray NaN sorts deterministically rather than panicking.
  - BenchConfig::cold_start docs spell out that it is forced `true`
    during the calibration pass.
  - Document the fresh-key window wraparound for
    write_new_key_fraction > 0 in total_benchmark_keys.

CLI polish
  - ANSI colour escapes are now emitted only when stdout is a TTY
    (cached behind OnceLock<bool>); piping benchmark output to a
    file or CI log no longer produces literal \x1b[...m sequences.

Docs
  - README: refreshed benchmark-output screenshot URL (kept the new
    user-attachments image from #6 over the in-repo placeholder
    that referenced a not-yet-committed file).
  - CONTRIBUTING.md: clarifications.
  - docs/assets/README.md: contributor guidance scaffolding.

Verification (local, all green):
  - cargo fmt --check
  - cargo clippy --all-targets --locked -- -D warnings
    (also from a fresh empty CARGO_HOME with no git creds)
  - cargo build --locked
  - cargo build --release --locked
  - cargo test --locked  (20 passed)
@caution-tape-bot
Copy link
Copy Markdown

👋 It seems that this PR is adding, or editing, a GitHub Action workflow that is not configured to use our custom GitHub Action runners.

For private repositories, the only supported way of running Action workflows is via the custom runners. You can learn more about the supported use cases in our GitHub Actions documentation.

To make sure that your workflow is using these runners you need to add the shopify-ubuntu-latest label to your workflow manifest file, or shopify-core-ubuntu-latest for Core repositories:

runs-on: shopify-ubuntu-latest

If your repository is intended to become Public later, please use runs-on: shopify-ubuntu-latest for now. The custom runners should function the same as ubuntu-latest public runners, though the exact Ubuntu version may vary. Please see the section below for more information about OS selection on premium runners.

If you run into any issues with the custom runners, please reach out in help-eng-infrastructure on Discourse.

Note for OS Versions

The shopify-ubuntu-latest runners can select from two different runner pools, which contain both Ubuntu 20.04 and Ubuntu 22.04 machines. For some workflows, this can cause issues.

To use Ubuntu 20.04 only, you can specify runs-on: shopify-ubuntu-20.04.

To use Ubuntu 22.04 only, you can specify runs-on: shopify-ubuntu-latest-2.


This comment was added by the GitHubActions::Workflow Caution Tape Bot rule.
View the source of this rule in Services DB

Warning

CautionTapeBot Deprecation Notice

CautionTapeBot is being deprecated and will be sunset in Q1 2026. Please plan to migrate your workflows.

📚 View full deprecation guide and migration instructions

@sam-obeid sam-obeid merged commit a809bdf into main May 4, 2026
4 checks passed
@sam-obeid sam-obeid deleted the fix/gate-neocache-behind-feature branch May 4, 2026 15:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant