Record Depot phase metrics

[ndizazzo]

Summary

• Upload 14-day telemetry records for native package, dry-image, and staged-image Depot phases.
• Record exact matrix identity, runner label, Depot project/build IDs, action timing, and context-size estimates.
• Keep unsupported cache, upload, CPU, and cost fields explicit; preserve the archive-first matrix and shared cache project.

Validation

• node --experimental-strip-types --test tests/release-workflow.test.ts
• node --experimental-strip-types scripts/image-matrix.ts validate
• actionlint .github/workflows/package-image-row.yml
• git diff --check

This PR intentionally adds measurement rather than speculative runner or concurrency tuning.

Summary by CodeRabbit

• New Features

  • Release packaging workflows now capture timing, context-size, build, runner, cache, and phase details for package and image operations.
  • Measurement artifacts are uploaded even when earlier workflow steps fail and retained for 14 days.

• Documentation

  • Added guidance on execution boundaries, matrix behavior, runner capacity, tuning constraints, and interpreting per-phase build telemetry.

• Tests

  • Added coverage validating expected phases, artifact contents, retention, and unavailable metrics.

[coderabbitai]

Warning

Review limit reached

@ndizazzo, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 44 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 4d07d2d7-79b9-41f8-bb7b-e4838f915ab0

📥 Commits

Reviewing files that changed from the base of the PR and between e264629 and a4aa469.

📒 Files selected for processing (3)

• .github/workflows/package-image-row.yml
• docs/matrix.md
• tests/release-workflow.test.ts

📝 Walkthrough

Walkthrough

The release workflow now records and uploads JSON measurements for native packaging, dry-image builds, and staged-image builds. Tests validate the evidence fields and retention settings. Documentation defines the execution matrix and Depot tuning boundaries.

Changes

Depot telemetry

Layer / File(s)	Summary
Per-phase measurement and artifact upload .github/workflows/package-image-row.yml, tests/release-workflow.test.ts	The workflow records timing and context size for all three phases. It emits phase-specific JSON with Depot, matrix, runner, cache, and resource fields, then uploads artifacts with 14-day retention. Tests validate these fields and labels.
Execution and tuning boundaries docs/matrix.md, docs/runner-capacity.md	The documentation describes the unchanged execution matrix, flow constraints, Depot evidence limits, runner and project boundaries, and retained per-phase records.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant PackageImageWorkflow
  participant Depot
  participant GitHubArtifactStorage
  PackageImageWorkflow->>Depot: Run package, dry-image, and staged-image phases
  Depot-->>PackageImageWorkflow: Return build and project metadata
  PackageImageWorkflow->>GitHubArtifactStorage: Upload phase measurement JSON

Loading

Possibly related PRs

• Mesh-LLM/mesh-packaging#16: Modifies the same workflow and release-workflow tests that this PR extends with telemetry.
• Mesh-LLM/mesh-packaging#17: Extends the same Depot-based package and image workflow with per-phase telemetry.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: recording metrics for Depot phases.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch agent/tune-depot-observability-packaging

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/package-image-row.yml:
- Line 196: The package-image workflow currently stringifies runner labels when
passing them to record-generation steps. Update RUNNER_LABEL and the
native-package, dry-image, and staged-image records to use JSON passthrough and
emit runner.labels as an array. Apply the corresponding workflow assertions or
fixtures in tests/release-workflow.test.ts so all affected records preserve the
original JSON array.

In `@docs/matrix.md`:
- Around line 56-79: Add a distinct statement in docs/matrix.md documenting
macOS GPU/runtime limitations, separate from the Linux package/image execution
flow described in the “Execution and tuning boundary” section. Keep the existing
Linux behavior and matrix details unchanged, and avoid implying that macOS
GPU/runtime support uses the container packaging pipeline.

In `@tests/release-workflow.test.ts`:
- Around line 106-134: Update the test named “each Depot phase emits bounded
tuning evidence without claiming unavailable metrics” to assert that every Depot
phase preserves runner labels as a JSON-array field and uses the jq --argjson
input path. Add these assertions inside the existing per-job loop, alongside the
other measurement-record checks, without changing the existing metric
expectations.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: b736e57f-5d2f-42ef-9523-61f8dddf02ce

📥 Commits

Reviewing files that changed from the base of the PR and between 1a444a1 and e264629.

📒 Files selected for processing (4)

• .github/workflows/package-image-row.yml
• docs/matrix.md
• docs/runner-capacity.md
• tests/release-workflow.test.ts