feat: add headless Apple harness - #156
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces a typed, headless Swift harness (putio-harness) that automates building, booting, launching, exercising, and capturing proof (screenshots/recordings + JSON manifest) for the iOS, paired watchOS, and tvOS shells, and wires it into repository scripts, mise tasks, CI, and contributor/agent documentation.
Changes:
- Add a new Swift package tool (
Tools/PutioHarness) implementing argument parsing, doctor checks, simulator lifecycle automation, proof artifact capture, and optional live adapters (putio/attach). - Update scripts/mise/CI/docs/templates to use the harness for doctor/build/proof flows and to require proof capture as part of the PR definition of done.
- Add a small app-surface “exercise scenario” hook via
SignedOutPresentationso the harness can reliably validate “rendered frame + exercise complete” across platforms.
Reviewed changes
Copilot reviewed 32 out of 32 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| Tools/PutioHarness/Tests/PutioHarnessKitTests/ProcessRunnerTests.swift | Adds coverage to ensure sensitive env vars can be stripped from child processes. |
| Tools/PutioHarness/Tests/PutioHarnessKitTests/ModelsTests.swift | Adds model-level tests for platform configuration, manifest encoding, doctor status, and runtime matching. |
| Tools/PutioHarness/Tests/PutioHarnessKitTests/ArgumentParserTests.swift | Adds parsing/validation tests for harness CLI invocations and error formatting behavior. |
| Tools/PutioHarness/Sources/PutioHarnessKit/SimulatorHarness.swift | Implements headless simctl-based build/boot/launch/exercise/proof capture and manifest emission. |
| Tools/PutioHarness/Sources/PutioHarnessKit/RepositoryContext.swift | Adds repo discovery + shared runtime/device JSON decoding and runtime/SKD compatibility logic. |
| Tools/PutioHarness/Sources/PutioHarnessKit/ProcessRunner.swift | Adds a reusable process runner (checked runs + async recording helper). |
| Tools/PutioHarness/Sources/PutioHarnessKit/Models.swift | Defines harness CLI models, output models, doctor report, and proof manifest/artifact schemas. |
| Tools/PutioHarness/Sources/PutioHarnessKit/LiveAdapters.swift | Adds optional adapters for putio live profile checks/fixture provisioning and attach publishing. |
| Tools/PutioHarness/Sources/PutioHarnessKit/HarnessService.swift | Wires parsed invocations to simulator/live implementations and formats results. |
| Tools/PutioHarness/Sources/PutioHarnessKit/Doctor.swift | Adds structured doctor checks for toolchain/workspace/runtimes and optional live lane tools. |
| Tools/PutioHarness/Sources/PutioHarnessKit/ArgumentParser.swift | Implements a dependency-free argument parser with strong validation and stable help text. |
| Tools/PutioHarness/Sources/PutioHarness/main.swift | Adds the executable entrypoint with JSON/text rendering and structured error output. |
| Tools/PutioHarness/Package.swift | Adds a new Swift package (executable + library + tests) for the harness. |
| scripts/verify.sh | Ensures doctor runs as part of verify before tests/build. |
| scripts/test.sh | Extends lint scope to Tools/ and runs harness package tests. |
| scripts/harness.sh | Adds a thin wrapper to run the harness via swift run --package-path Tools/PutioHarness. |
| scripts/harness-ci.sh | Adds a CI-focused headless iOS proof runner with stable run-id selection. |
| scripts/doctor.sh | Switches doctor implementation to the harness doctor command. |
| scripts/build.sh | Switches build implementation to the harness build command (--platform all). |
| scripts/bootstrap.sh | Adds a bootstrap script that generates the workspace then runs harness doctor. |
| README.md | Links to harness documentation and contributing guidance entrypoints. |
| Packages/PutioCore/Tests/PutioCoreTests/SignedOutPresentationTests.swift | Adds tests covering the harness scenario switch and exercise detection. |
| Packages/PutioCore/Sources/PutioCore/SignedOutPresentation.swift | Adds harness scenario selection and an exercise marker signal to support runtime proof. |
| mise.toml | Adds harness and harness-ci tasks and updates bootstrap/verify wiring. |
| docs/HARNESS.md | Adds a full harness contract and command inventory, including proof layout and live lane boundaries. |
| CONTRIBUTING.md | Documents JSON doctor output and recommends harness exercise/proof for runtime-sensitive changes. |
| Apps/watchOS/Sources/PutioWatchApp.swift | Uses harness scenario to present/mark “exercise complete” for watchOS. |
| Apps/tvOS/Sources/PutioTVApp.swift | Uses harness scenario to present/mark “exercise complete” for tvOS. |
| Apps/iOS/Sources/PutioApp.swift | Uses harness scenario to present/mark “exercise complete” for iOS. |
| AGENTS.md | Adds harness commands and constraints to agent workflow guidance. |
| .github/workflows/ci-next.yml | Adds a CI step to run the headless iOS proof subset. |
| .github/pull_request_template.md | Updates validation checklist to require harness exercise + proof + separate publish step. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
There was a problem hiding this comment.
All reported issues were addressed across 32 files
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
There was a problem hiding this comment.
All reported issues were addressed across 8 files (changes from recent commits).
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
There was a problem hiding this comment.
All reported issues were addressed across 6 files (changes from recent commits).
Tip: Review your code locally with the cubic CLI to iterate faster.
Re-trigger cubic
Summary
attachpublishing adapter, and an idempotentdevs-fe-autolive-fixture adapterRefs #146
Reviewer Guide
SimulatorHarness.swiftdocs/HARNESS.md, thenArgumentParser.swift,SimulatorHarness.swift, andLiveAdapters.swiftVisual Aids
Validation
mise run verifymise run harness -- proof --platform all --run-id bot-review-6f42782 --record-seconds 3 --output json6f42782with artifact sizes and SHA-256 digestsdevs-fe-autoprofile readiness passed; live fixture was created through dry-run then reused idempotentlygpt-5.6-sol, high effort) returned clean after every accepted fix; the focused final review of6f42782had zero findingsmise install→ bootstrap/doctor → all-platform headless proof after doctor identified the missing tvOS runtime and it was provisionedNext CIand Cubic AI review passed on final commit6f42782; all 14 GitHub review threads are resolvedSanity Checks
attachBenchmarks
mise run verify: approximately 20 seconds on the MacBook after warm cachesNotes