Claude Code + MCP + macOS accessibility API: the wiring that actually works

Four pieces. (1) Claude Code is Anthropic's terminal CLI; it speaks MCP (Model Context Protocol) to local servers over stdio. (2) An MCP server is just a binary that reads JSON-RPC 2.0 from stdin and writes responses to stdout. (3) The macOS Accessibility API is AXUIElement: a read/write tree of every visible UI element in any running app, plus CGEvent for synthesizing mouse and keyboard input. (4) mcp-server-macos-use is a Swift binary that bridges 2 and 3: it exposes 9 MCP tools (registered in the allTools array at Sources/MCPServer/main.swift:1482) that read the AX tree of a target app, post CGEvent inputs, and return the post-action tree. Wiring all four together gives Claude Code real hands on any macOS app, no screenshot OCR or pixel matching required.

macOS TCC (Transparency, Consent, Control) keys Accessibility permission on the *responsible* process for a request, not the literal calling executable. When mcp-server-macos-use calls AXUIElementCopyAttributeValue, the kernel asks 'who is this child accountable to?' and walks up the parent chain. It finds claude (a node process), then the shell, then the terminal app that owns the PTY. The terminal is the long-lived bundle-id'd application TCC has on file, so the Accessibility prompt asks about iTerm/Terminal/Ghostty/VS Code, not about /usr/local/bin/mcp-server-macos-use. Symptom of getting this wrong: you toggle the MCP binary on in System Settings, restart, and the first AX call still returns errAXAPIDisabled. Fix: revoke the binary, grant your terminal app, restart the terminal (TCC caches the decision per-PID).

Yes. The `--` separator hands everything after to the spawn args, so claude will launch `npx -y mcp-server-macos-use` over stdio when the server is requested. The `-y` on npx auto-accepts the install prompt. The npm package's postinstall hook runs `xcrun swift build -c release` to produce the native binary, so the only host requirement is the Xcode Command Line Tools (no full Xcode needed). If you'd rather not have npm in the path, clone the repo, `swift build -c release`, and pass an explicit path: `claude mcp add macos-use -- /Users/you/mcp-server-macos-use/.build/release/mcp-server-macos-use`.

Nine, at Sources/MCPServer/main.swift:1482. The full set: open_application_and_traverse, click_and_traverse, type_and_traverse, press_key_and_traverse, scroll_and_traverse, set_value_and_traverse, press_ax_and_traverse, set_selected_and_traverse, refresh_traversal. The README highlights five; the llms.txt mentions six. Three of the nine (set_value, press_ax, set_selected) are escape hatches that write AX attributes directly instead of going through CGEvent. They land on fields that reject synthesized input, such as secure-input password fields, sudo prompts, and 1Password unlock, where CGEvent-based clicks and types silently no-op. If you've ever had a 'the click happened, the field is still empty' bug, that's the one. press_ax_and_traverse calls kAXPressAction directly on the element, which the system accepts because it looks like the app calling itself.

Take macos-use_click_and_traverse as the canonical shape. (1) The handler engages InputGuard, which installs a head-insert CGEventTap that drops hardware keyboard and mouse events for the duration of the call (Esc, plain, with no modifiers, is the one exception; it cancels the call). (2) It reads the target app's AX tree once for the 'before' state. (3) It posts a CGEvent left-mouse-down + left-mouse-up at (x + w/2, y + h/2), centering on whatever element you asked for. (4) It reads the AX tree again for the 'after' state. (5) It diffs the two trees and returns a compact summary plus a file path. The full traversal lands as a text file under /tmp/macos-use/, one line per element, `[AXRole] "label" x:N y:N w:W h:H visible`. The compact summary is what the LLM sees in its context window; the file is what you grep when something goes wrong.

Two things. First, every disruptive tool call engages InputGuard before posting any CGEvent. The guard is a 30-second watchdog (InputGuard.swift, line 24: `var watchdogTimeout: TimeInterval = 30`) that force-disengages the input tap even if the server hangs mid-call. Second, the guard listens for plain Esc with no modifiers, and if you press it the in-flight tool call throws InputGuardCancelled and returns control to you. The cursor position is also saved before each action and restored after, so the visible mouse pointer doesn't jump around your screen.

Any MCP-compatible client over stdio. Claude Desktop, Cursor, VS Code (with Copilot's MCP host), Cline, Windsurf; they all speak the same protocol. The config shape differs (Claude Desktop wants the `mcpServers` block in `claude_desktop_config.json`; Claude Code wants `claude mcp add`; Cursor wants its own settings.json entry), but the server binary is the same. The TCC trap is the same on every host: grant Accessibility to whichever app spawned the MCP server, not to the server binary itself. If you launch the host from Spotlight that's the host's bundle ID; if you launch the host from a terminal that's the terminal's bundle ID.

Any app the Accessibility API can see, which is most of them, with the exception of apps that explicitly opt out (a small list, mostly DRM-protected video players). Concrete examples: 'open Calendar, find next Tuesday 2pm, create an event titled X' (open_application_and_traverse + click + type + press Return), 'in Finder, navigate to ~/Downloads and right-click the first PDF' (open + click with rightClick: true), 'find the unread Slack message in the General channel and read it' (click + refresh_traversal to read the new pane). The set_value tool gives you a write primitive that bypasses CGEvent, which is what you reach for when an app has a text field that rejects synthesized typing.

Every tool call writes a flat text file to /tmp/macos-use/<timestamp>_<tool>.txt and (for disruptive actions) a PNG screenshot of the target window at the same prefix. The text file is one element per line: `[AXButton (button)] "Submit" x:680 y:520 w:80 h:30 visible`. Diff lines (for the after-state tree) use `+` (added), `-` (removed), `~` (modified) prefixes. Don't read the full file into an LLM's context; grep for the element you care about: `grep -n "AXTextField" /tmp/macos-use/*_click_and_traverse.txt`. The MCP summary returned to the model includes counts and up to three notable text changes, which is enough for the model to decide what to do next.

Open source at github.com/mediar-ai/mcp-server-macos-use under BSL 1.1 (Business Source License 1.1; converts to a permissive open-source license after a fixed period). Written in Swift: 2056 lines in Sources/MCPServer/main.swift (the JSON-RPC handler, tool registry, AX traversal, CGEvent injection) and 355 lines in Sources/MCPServer/InputGuard.swift (the CGEventTap input arbiter and watchdog). Distributed via npm so the install is one command and the postinstall hook builds the Swift release binary against your local Xcode toolchain.