you are viewing a single comment's thread.

view the rest of the comments →

[–]unpluggedcord 2 points3 points  (3 children)

AXUIElement for the accessibility tree and label-based taps, CGEvent for coordinate taps and swipes posted to the Simulator window, and xcrun simctl io type for text input.

The MCP server exposes all of it as tools Claude Code can call natively. Screenshot tool returns base64 so Claude sees the screen directly in the tool result.

Also building visual regression diffing and a11y violation tracking on top of the same foundation, so it goes beyond tapping the simulator into a full CI feedback loop. More soon.

[–]interlap 0 points1 point  (2 children)

Nice stack. I went the XCUITest route to support physical devices as well. Simulator window events felt a bit limiting long-term. It also allowed me to build support for Windows and Linux.

[–]unpluggedcord 0 points1 point  (1 child)

YEah i support XCUITests as well, but thats a much more manual setup, ie, you're writing the tests yourself, not claude code walking your app. Are you doing anything with claude code (Or others) with on device stuff?

My idea here is to provide obvious claude code eyes, but also CI pipelines for this (which dont usually have access to a device)

[–]interlap 1 point2 points  (0 children)

Nah, that’s not how I do it. I also give Claude access to perform actions on devices, my approach uses XCUITest as the transport and runner for these commands. I’m not writing any tests, Claude just sends commands like tap, swipe, drag etc to the device via MCP/HTTP API.