feat: add ios_screenshot MCP tool#30
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new iOS screenshot capability to VisionTest by extending the iOS automation server with a ui.screenshot JSON-RPC method and exposing an ios_screenshot MCP tool in the Kotlin server that decodes and writes the PNG to disk.
Changes:
- iOS (Swift) JSON-RPC: add
ui.screenshotdispatch plusScreenshotResultmodel and bridge implementation. - MCP server (Kotlin): add
IOSAutomationClient.screenshot()and registerios_screenshottool (path resolution + base64 decode + file write). - Specs/docs/tests: add OpenSpec capability docs, CLAUDE tool table row, and unit tests in both Swift and Kotlin.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| openspec/specs/ios-screenshot/spec.md | New spec describing ios_screenshot tool + ui.screenshot JSON-RPC method requirements. |
| openspec/changes/archive/2026-04-16-add-ios-screenshot/tasks.md | Archived implementation checklist for the change. |
| openspec/changes/archive/2026-04-16-add-ios-screenshot/specs/ios-screenshot/spec.md | Archived copy of the new spec. |
| openspec/changes/archive/2026-04-16-add-ios-screenshot/proposal.md | Archived proposal describing motivation/scope for the new tool. |
| openspec/changes/archive/2026-04-16-add-ios-screenshot/design.md | Archived design notes covering transport/path decisions and risks. |
| openspec/changes/archive/2026-04-16-add-ios-screenshot/.openspec.yaml | OpenSpec metadata for the archived change. |
| ios-automation-server/IOSAutomationServerUITests/Server/JsonRpcServer.swift | Adds ui.screenshot method dispatch to the Swift JSON-RPC server. |
| ios-automation-server/IOSAutomationServerUITests/Models/AutomationModels.swift | Introduces ScreenshotResult and toDictionary() serialization. |
| ios-automation-server/IOSAutomationServerUITests/Bridge/XCUITestBridge.swift | Implements screenshot capture via XCUIScreen.main.screenshot().pngRepresentation. |
| ios-automation-server/IOSAutomationServerTests/AutomationModelsTests.swift | Adds unit tests for ScreenshotResult.toDictionary(). |
| app/src/test/kotlin/com/example/visiontest/tools/IOSScreenshotToolTest.kt | Adds Kotlin unit tests for path resolution, decoding, error cases, and file writing. |
| app/src/main/kotlin/com/example/visiontest/tools/IOSAutomationToolRegistrar.kt | Registers ios_screenshot tool and implements capture + write logic. |
| app/src/main/kotlin/com/example/visiontest/ios/IOSAutomationClient.kt | Adds screenshot() client method delegating to sendRequest("ui.screenshot"). |
| CLAUDE.md | Documents the new ios_screenshot tool in the iOS tools table. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ios_screenshotMCP tool that captures the current iOS simulatordisplay as a PNG via XCUITest and saves it to the host filesystem.
./screenshots/ios_screenshot_<yyyyMMdd_HHmmss>.pngresolved against the MCP server's CWD (the user's project when launched
by a coding agent), overridable via an optional
outputPathparameter.ui.screenshotJSON-RPC method returns a base64-encoded PNG; the Kotlin tool decodes
and writes to disk.
Changes
iOS automation server (Swift)
ScreenshotResultmodel withtoDictionary()inAutomationModels.swiftXCUITestBridge.screenshot()usingXCUIScreen.main.screenshot().pngRepresentationui.screenshotcase added toJsonRpcServer.executeMethodMCP server (Kotlin)
IOSAutomationClient.screenshot()suspend functionios_screenshottool registered inIOSAutomationToolRegistrarwithtimeoutMs = 30000, CWD-relative default path, auto-created parentdirectories, and graceful handling of malformed responses, missing
pngBase64(outdated-bundle hint), and server-not-running.Documentation
CLAUDE.mdopenspec/changes/archive/2026-04-16-add-ios-screenshot/openspec/specs/ios-screenshot/spec.mdDesign notes
preserve the single-method dispatch pattern. For typical iPhone-sized
simulators the payload is well under 3 MB base64.
semantics are unambiguous — the simulator sandbox would resolve paths
differently from the host.
VISIONTEST_DIR, because acoding agent taking a screenshot is almost always working on a user
project and screenshots should land there.
ui.screenshotmethod ships in a new bundle release. Users on an older tarball will
get a clear "outdated bundle" error from the tool until they update.
Test plan
./gradlew :app:test— all Kotlin tests pass including 10 newcases in
IOSScreenshotToolTest(path resolution, base64 decoding,parent-dir creation, server-not-running,
success:false, missingpngBase64, missingresult, malformed JSON).xcodebuild test ... -only-testing:IOSAutomationServerTests—all 69 tests pass including 2 new
ScreenshotResult.toDictionary()cases (tested on iPhone 17 simulator).
simulator, invoked
ios_screenshot, confirmed the PNG opens andrenders the simulator display at native resolution.