Skip to content

feat: add headless Apple harness - #156

Merged
altaywtf merged 22 commits into
nextfrom
feat/146-agent-harness
Aug 10, 2026
Merged

feat: add headless Apple harness#156
altaywtf merged 22 commits into
nextfrom
feat/146-agent-harness

Conversation

@altaywtf

@altaywtf altaywtf commented Aug 9, 2026

Copy link
Copy Markdown
Member

Summary

  • Adds a dependency-free Swift harness with validated iOS, paired watchOS, and tvOS build, boot, launch, exercise, screenshot, recording, and proof commands
  • Makes simulator automation headless by default with isolated ephemeral devices, Watch/iPhone pairing, rendered-frame assertions, cleanup, concise errors, and structured JSON output
  • Adds structured host doctor checks, deterministic proof manifests, a separate attach publishing adapter, and an idempotent devs-fe-auto live-fixture adapter
  • Wires the harness into mise, CI, contributor guidance, agent guidance, and the pull-request definition of done

Refs #146

Reviewer Guide

  • Highest risk: simulator lifecycle and rendered-frame assertions in SimulatorHarness.swift
  • Start with docs/HARNESS.md, then ArgumentParser.swift, SimulatorHarness.swift, and LiveAdapters.swift
flowchart LR
  M["mise commands"] --> H["Typed Swift harness"]
  H --> X["Xcode and Tuist"]
  H --> S["Headless simctl adapter"]
  H --> P["Proof artifacts and manifests"]
  H --> L["Optional putio live profile"]
  P --> A["Explicit attach publish command"]
Loading

Visual Aids

Validation

  • mise run verify
  • mise run harness -- proof --platform all --run-id bot-review-6f42782 --record-seconds 3 --output json
  • iOS, paired watchOS, and tvOS processes stayed alive and rendered visible frames
  • H.264 recordings validated for all three platforms
  • Proof manifests point to commit 6f42782 with artifact sizes and SHA-256 digests
  • devs-fe-auto profile readiness passed; live fixture was created through dry-run then reused idempotently
  • Invalid platform and missing-workspace failures returned structured actionable errors
  • Codex autoreview (gpt-5.6-sol, high effort) returned clean after every accepted fix; the focused final review of 6f42782 had zero findings
  • Simulator.app remained closed and every harness-created device was deleted
  • Fresh macOS devbox clone completed mise install → bootstrap/doctor → all-platform headless proof after doctor identified the missing tvOS runtime and it was provisioned
  • GitHub Actions Next CI and Cubic AI review passed on final commit 6f42782; all 14 GitHub review threads are resolved

Sanity Checks

  • The first Watch proof produced a black frame; decoded-pixel readiness rejected that false positive after the assertion was strengthened
  • watchOS proof builds and installs the iOS companion, creates and activates a fresh pair, wakes the Watch display, and launches the Watch app
  • Capture never uploads; publishing is a separate command that delegates to attach
  • The devbox proof used the same branch commit and commands as the MacBook; no host-specific harness changes were required

Benchmarks

  • Full local mise run verify: approximately 20 seconds on the MacBook after warm caches
  • Exact-SHA all-platform proof: approximately 90 seconds with fresh ephemeral devices

Notes

  • GitHub Actions runs the same headless iOS proof subset after the canonical verify gate
  • Simulator proof does not claim physical-device behavior, background execution, production signing, remote-control behavior, or Watch hardware behavior
  • Brand fonts remain explicitly unconfigured until Bundle brand typography in the new app targets #126
  • App session injection, device-code approval, and richer live fixture creation remain follow-up work behind their owning CLI and app contracts

@altaywtf
altaywtf marked this pull request as ready for review August 10, 2026 08:18
Copilot AI lite review requested due to automatic review settings August 10, 2026 08:18

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a typed, headless Swift harness (putio-harness) that automates building, booting, launching, exercising, and capturing proof (screenshots/recordings + JSON manifest) for the iOS, paired watchOS, and tvOS shells, and wires it into repository scripts, mise tasks, CI, and contributor/agent documentation.

Changes:

  • Add a new Swift package tool (Tools/PutioHarness) implementing argument parsing, doctor checks, simulator lifecycle automation, proof artifact capture, and optional live adapters (putio / attach).
  • Update scripts/mise/CI/docs/templates to use the harness for doctor/build/proof flows and to require proof capture as part of the PR definition of done.
  • Add a small app-surface “exercise scenario” hook via SignedOutPresentation so the harness can reliably validate “rendered frame + exercise complete” across platforms.

Reviewed changes

Copilot reviewed 32 out of 32 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
Tools/PutioHarness/Tests/PutioHarnessKitTests/ProcessRunnerTests.swift Adds coverage to ensure sensitive env vars can be stripped from child processes.
Tools/PutioHarness/Tests/PutioHarnessKitTests/ModelsTests.swift Adds model-level tests for platform configuration, manifest encoding, doctor status, and runtime matching.
Tools/PutioHarness/Tests/PutioHarnessKitTests/ArgumentParserTests.swift Adds parsing/validation tests for harness CLI invocations and error formatting behavior.
Tools/PutioHarness/Sources/PutioHarnessKit/SimulatorHarness.swift Implements headless simctl-based build/boot/launch/exercise/proof capture and manifest emission.
Tools/PutioHarness/Sources/PutioHarnessKit/RepositoryContext.swift Adds repo discovery + shared runtime/device JSON decoding and runtime/SKD compatibility logic.
Tools/PutioHarness/Sources/PutioHarnessKit/ProcessRunner.swift Adds a reusable process runner (checked runs + async recording helper).
Tools/PutioHarness/Sources/PutioHarnessKit/Models.swift Defines harness CLI models, output models, doctor report, and proof manifest/artifact schemas.
Tools/PutioHarness/Sources/PutioHarnessKit/LiveAdapters.swift Adds optional adapters for putio live profile checks/fixture provisioning and attach publishing.
Tools/PutioHarness/Sources/PutioHarnessKit/HarnessService.swift Wires parsed invocations to simulator/live implementations and formats results.
Tools/PutioHarness/Sources/PutioHarnessKit/Doctor.swift Adds structured doctor checks for toolchain/workspace/runtimes and optional live lane tools.
Tools/PutioHarness/Sources/PutioHarnessKit/ArgumentParser.swift Implements a dependency-free argument parser with strong validation and stable help text.
Tools/PutioHarness/Sources/PutioHarness/main.swift Adds the executable entrypoint with JSON/text rendering and structured error output.
Tools/PutioHarness/Package.swift Adds a new Swift package (executable + library + tests) for the harness.
scripts/verify.sh Ensures doctor runs as part of verify before tests/build.
scripts/test.sh Extends lint scope to Tools/ and runs harness package tests.
scripts/harness.sh Adds a thin wrapper to run the harness via swift run --package-path Tools/PutioHarness.
scripts/harness-ci.sh Adds a CI-focused headless iOS proof runner with stable run-id selection.
scripts/doctor.sh Switches doctor implementation to the harness doctor command.
scripts/build.sh Switches build implementation to the harness build command (--platform all).
scripts/bootstrap.sh Adds a bootstrap script that generates the workspace then runs harness doctor.
README.md Links to harness documentation and contributing guidance entrypoints.
Packages/PutioCore/Tests/PutioCoreTests/SignedOutPresentationTests.swift Adds tests covering the harness scenario switch and exercise detection.
Packages/PutioCore/Sources/PutioCore/SignedOutPresentation.swift Adds harness scenario selection and an exercise marker signal to support runtime proof.
mise.toml Adds harness and harness-ci tasks and updates bootstrap/verify wiring.
docs/HARNESS.md Adds a full harness contract and command inventory, including proof layout and live lane boundaries.
CONTRIBUTING.md Documents JSON doctor output and recommends harness exercise/proof for runtime-sensitive changes.
Apps/watchOS/Sources/PutioWatchApp.swift Uses harness scenario to present/mark “exercise complete” for watchOS.
Apps/tvOS/Sources/PutioTVApp.swift Uses harness scenario to present/mark “exercise complete” for tvOS.
Apps/iOS/Sources/PutioApp.swift Uses harness scenario to present/mark “exercise complete” for iOS.
AGENTS.md Adds harness commands and constraints to agent workflow guidance.
.github/workflows/ci-next.yml Adds a CI step to run the headless iOS proof subset.
.github/pull_request_template.md Updates validation checklist to require harness exercise + proof + separate publish step.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread Tools/PutioHarness/Sources/PutioHarnessKit/ProcessRunner.swift
Comment thread Tools/PutioHarness/Sources/PutioHarnessKit/SimulatorHarness.swift
Comment thread Tools/PutioHarness/Sources/PutioHarnessKit/Doctor.swift

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 32 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread Tools/PutioHarness/Sources/PutioHarness/main.swift
Comment thread scripts/verify.sh
Comment thread scripts/doctor.sh
Comment thread Tools/PutioHarness/Sources/PutioHarnessKit/HarnessService.swift
Comment thread Tools/PutioHarness/Sources/PutioHarnessKit/LiveAdapters.swift
Comment thread Tools/PutioHarness/Sources/PutioHarnessKit/HarnessService.swift Outdated
Comment thread docs/HARNESS.md Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 8 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread Tools/PutioHarness/Sources/PutioHarnessKit/SimulatorHarness.swift Outdated
Comment thread Tools/PutioHarness/Sources/PutioHarnessKit/Doctor.swift Outdated

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 6 files (changes from recent commits).

Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic

Comment thread Tools/PutioHarness/Sources/PutioHarnessKit/HarnessService.swift Outdated
Comment thread Tools/PutioHarness/Sources/PutioHarnessKit/SimulatorHarness.swift
@altaywtf
altaywtf merged commit 6b7b80c into next Aug 10, 2026
4 checks passed
@altaywtf
altaywtf deleted the feat/146-agent-harness branch August 10, 2026 10:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants