Get Moshi
Part of Moshi · iOS · Android

Give your agent a device.

Mata is Moshi's device layer. It turns every iOS Simulator and Android emulator on a Mac into a network device. Video goes out, touch comes in, and the screen reads as JSON. One port. No SDK, no keys.

Save trail buttonopus-5-5 ~/projects/trails-app save-trail
Save trail buttontrails-app · opus-5-5
Add a Save trail button to the trail page, then test it end to end on the simulator.
Chat via Moshi…
iPhone 17 Pro
watch & drive from

Native on your phone and your computer. In the browser everywhere else.

  • Phone & tabletiOS · AndroidNative app
  • DesktopmacOS · Windows · LinuxNative app
  • WebAny browserClient embedded

~60 fps H.264HTTP + WebSocketNo accounts

01 · the cli

Observe, act, verify. In plain commands.

Read the screen as data, find the element by its label or id, act on it, and check the result — no pixel guessing. Coordinates are 0–1 fractions of the screen, so the same tap works on every model.

Every command is one HTTP request to the Mata server, so anything that can open a socket can drive a device: curl $MATA_URL/sim/booted/ui. The v1 protocol is frozen — it only ever grows.

mata · from any machine on your tailnet
# what's on screen — as data
$ mata ui
{ "type": "Button",
  "AXLabel": "Save trail",
  "AXUniqueId": "saveTrail",
  "frame": { "x": 24, "y": 612,
             "width": 354, "height": 50 } }

# act on it: the centre of its frame
$ mata tap 0.5 0.73
$ mata type "Great views"; mata key enter

# verify — read it back, or look
$ mata ui   → "AXLabel": "Saved"
$ mata screenshot -o after.jpg
02 · every verb

One CLI. Every device. Same verbs.

Observe, act, verify: everything an agent needs to use an app, as plain commands. Point them at an iPhone or an Android emulator — only the device id changes.

iPhone 17 Prolive · H.264

A ready-made skill teaches Claude and Codex the whole loop.

03 · how fast

Fast enough to feel like the device in your hand.

Video comes straight off the framebuffer, and touch lands in tens of milliseconds. Android emulators (API 36 here) only ship a software video encoder, so Mata takes their frames and encodes them on the Mac's GPU — the same path as the iOS Simulator.

Smoothness during a swipeFrames per second — higher is smoother; 60 looks like a real phone
iPhone simulator · for reference67 fpsa frame every 15 ms
Emulator · Mata on the Apple GPU59 fpsa frame every 17 ms
Emulator · Mata on a software GPU33 fpsa frame every 30 ms
Emulator · its own encoder16 fpsa frame every 63 ms
Response timeFrom the command to the result
~45 ms
Tap → new pixelsiOS Simulator
~80 ms
Tap → new pixelsAndroid emulator
~45 ms
Screenshotwarm, either platform
~1 s
Whole accessibility treeiOS · ~2.5 s on Android

Measured on an M4 Pro Mac mini with an API 36 emulator and an iPhone 17 Pro simulator. The Android tree comes from uiautomator.

04 · safe by default

No keys. The network is the lock.

Mata answers loopback, your LAN and your Tailscale peers — nobody else. No accounts, no relay, nothing in anyone's cloud. Host and Origin checks stop DNS rebinding and cross-site requests.

  • 127.0.0.1
  • 192.168.0.0/16
  • 100.64.0.0/10
  • *.ts.net
05 · the other end of the wire

Drive it yourself, or watch your agent drive.

Moshi puts the same device on your phone and your desktop: native H.264 at 60 fps and real touch, right next to your agent's session. Try the build yourself, or watch the agent test it — and take over with a tap when it gets stuck. How it works in Moshi

$curl -fsSL https://getmoshi.app/install.sh | sh

Mata comes with moshi-hook on macOS — install guide

Xcode 27 input from idb (MIT) · iOS capture from serve-sim (Apache-2.0) · Android capture from scrcpy (Apache-2.0)