cankolabuilds.com

How I Built a Codex Remote Control
for iPhone

I used the Codex Mac app to build a native SwiftUI iPhone app that talks back to it. Now I steer 10+ projects from my phone, and finished apps install back over Tailscale OTA.

10+
apps steered from phone
0
terminal windows
OTA
deploys via Tailscale
The Story

I'd been using the Codex Mac app for side projects and it was absurdly good. GPT-5.3 on medium thinking was solving things I expected to take hours. But the Mac was on my desk. And I wasn't always at my desk.

The pattern kept repeating: I'd think of a fix while making coffee, or want to check a build while waiting in line. By the time I got back to the Mac, the momentum was gone. I had 10 projects — a stocks app, a video frame extractor, a music app, a pixel art game — and they were all stuck whenever I walked away.

So one night I asked Codex a question: can you build me an iPhone app that talks back to you?

The Codex Mac app has something called app-server — the same protocol surface that powers the VS Code extension. Threads, turns, approvals, streaming output, all exposed over WebSocket. If VS Code can talk to it, an iPhone app can too.

That night, I used the Codex Mac app to build a Codex iPhone app. The tool built its own remote control.

I can finally build features for 10 apps at a time while on a 🚽 — directly from my phone — while it gets built on my Mac mini.

The first time I sent a prompt from my phone, watched Codex stream its response live, saw it build the app, and then installed the finished build back on the same phone I was holding — that was the moment. No cables. No Xcode Play button. Just a tap on "Install" and the app appeared on my home screen.

Now every gap in my day — waiting for coffee, sitting in a car, five minutes between meetings — is enough to review a thread, dictate a change, approve an action, and test the result. I went from shipping a couple of side-project builds per week to shipping something almost every day.

Not because I code more. Because I never have to wait until I'm at my desk.

The system now has its own canonical architecture reference, a runbook of every failure and fix, an operational checklist, and a release automation roadmap. It's not a hack anymore — it's infrastructure I rely on every day.

Fun fact — the OpenClaw founder recently shipped a fix for one of the complaints on Twitter for OpenClaw while he was at the barbershop. I think this will become normal soon.

📱

Pick a thread

Choose a project

🖥️

Mac builds

Codex executes

⬇️

Install back

OTA to your phone

Full Walkthrough

Add dark mode to an app — from your phone

Six real screens. One feature request. Watch Codex edit Swift, build the app, and install it back.

Codex iPhone to Mac Relay
Control Codex Mac on iPhone
Connected
📈
Stocks - ATH
🌴
pixelpalms-ios
🎬
FirstFrameLastFrame
✉️
Mail
🎵
Chetan Music
🔊
video-to-audio
🔗
kimiintegration
pixelpalms-ios
src/pixelpalms-ios · Connected
Add a dark mode toggle to the Settings screen. Use @AppStorage to persist the preference and apply .preferredColorScheme to the root view.
I'll add a dark mode toggle to SettingsView.swift using @AppStorage, then apply the color scheme modifier in your root ContentView. Let me check the current file structure first.
gpt-5.3-codex · Medium
pixelpalms-ios
Codex is working...
I found SettingsView.swift and ContentView.swift. I'm adding the dark mode toggle now.
📝 SettingsView.swift +14 −2
+ @AppStorage("darkMode") var darkMode = false
 
+ Toggle("Dark Mode", isOn: $darkMode)
+ .tint(.green)
📝 ContentView.swift +3 −1
+ @AppStorage("darkMode") var darkMode
 
+ .preferredColorScheme(
+ darkMode ? .dark : .light)
Editing 2 files...
pixelpalms-ios
Waiting for approval
Changes ready. 2 files modified. Approve to apply.
Apply file changes?
SettingsView.swift +14 −2
ContentView.swift +3 −1
Approve
Deny
After approval I'll build and publish the OTA install automatically.
Waiting for your decision...
pixelpalms-ios
Building...
Changes applied. Running xcodebuild for device...
Building pixelpalms-ios 0%
Archiving... Exporting IPA... Publishing via tailscale serve
✓ Compile Swift sources (47 files)
✓ Link PixelPalmsApp
✓ Sign with dev profile
✓ Create .xcarchive
✓ Export .ipa (ad-hoc)
✓ Generate manifest.plist
✓ Published on chetans-mac-mini.ts.net
Preparing installable build...
Dark Mode Toggle — Ready
pixelpalms-ios · OTA build published
Feature added
Dark mode toggle in Settings
Files changed
SettingsView.swift · ContentView.swift
Delivered via
chetans-mac-mini.tail4f7e57.ts.net
Install on This iPhone
Tap Install → test dark mode on this device
① Pick a project
Architecture

The full loop, visualized

Four nodes. One circle. Everything stays inside your Tailscale network.

📱
iPhone
SwiftUI client · Dictation
Prompts · Approvals · Attachments
🔀
Node Bridge
WebSocket proxy
History trimming
Payload shaping
:4600
🖥️
Mac mini
codex app-server
Repos · Xcode · Git · asc
xcodebuild · Shells
:4500
📡
Tailscale OTA
IPA export + serve
itms-services:// install
Private HTTPS delivery
Step 1
Send prompt
iPhone → Bridge
Step 2
Execute
Bridge → Mac
Step 3
Build + Export
Mac → OTA server
Step 4
Install
OTA → iPhone
Step 1
turn/start
Send text, images, files to a thread
Step 2
item/agentMessage/delta
Stream live output back to phone
Step 3
xcodebuild → export
Archive, export IPA, host on Mac
Step 4
tailscale serve → install
HTTPS OTA delivery to iPhone

📱 On iPhone

  • SwiftUI native app
  • iOS dictation input
  • Camera/file picker
  • Auto-reconnect logic
  • OTA install trigger

🔀 On Bridge

  • WebSocket normalization
  • History trimming (12 items)
  • Text truncation (4K chars)
  • Upload forwarding
  • iOS compatibility layer

🖥️ On Mac

  • codex app-server
  • All repos & git state
  • Xcode builds & archives
  • Shell execution
  • asc (App Store Connect)
  • launchd persistence

📡 Tailscale Layer

  • Private mesh network
  • MagicDNS resolution
  • HTTPS via tailscale serve
  • Works off-network
  • No exposed ports
Features

Everything the app does

Color-coded by dependency: what needs the app-server, what's pure iOS, what the bridge handles.

app-server

Thread Management

List, create, resume, fork, archive, search threads. Full JSON-RPC lifecycle.

app-server

Live Streaming Output

See Codex thinking character-by-character via item/agentMessage/delta.

app-server

Approvals

Accept or deny file changes and shell commands from your phone.

app-server

Model & Reasoning Control

Select model, set reasoning effort (low → xhigh). Populated via model/list.

app-server

Context Window Status

Token usage, manual compaction trigger via thread/compact/start.

app-server

Project CWD Switching

Set working directory per thread. Save multiple project paths.

ios native

Dictation Input

iOS speech-to-text. Talk to your Mac from anywhere. No server dependency.

ios native

Photo & File Attachments

Send photos, videos, files from camera roll into a Codex turn.

ios native

Auto-Reconnect

Exponential backoff (20s max), foreground re-sync, survives Mac sleep.

bridge

History Trimming

12 recent items, chat-relevant types only, 4K char limit. Mobile stays fast.

hybrid

OTA Install via Tailscale

Archive → export IPA → tailscale serve → Safari install. Works away from desk.

hybrid

Xcode Device Deploy

xcodebuild + xcrun devicectl. Build and install when phone is connected.

hybrid

Auto-Commit & Push

Automatically commit after Codex changes. Optional auto-push. Git stays clean.

hybrid

Auto-Deploy After Turn

Build + deploy after every turn completes. Change → build → install, zero taps.

app-server

Create Projects

Create folders and iOS app projects on Mac from your phone.

ios native

launchd Persistence

Both services run as user agents. Auto-restart on crash. Zero terminal windows.

Engineering

What was actually hard

01

Phone networking isn't localhost

127.0.0.1 on iPhone points to the phone itself. Every connection returned -1004.

→ Tailscale private mesh
02

iOS WebSockets are picky

URLSessionWebSocketTask had handshake incompatibilities. Connections died with -1005.

→ Node bridge on :4600
03

Thread history kills mobile

Full reads exceeded iOS WS message limits. Code=40 "Message too long".

→ Bridge trims to 12 items, 4K chars
04

Mac sleep kills everything

Locking the Mac interrupted transport. Dropped sockets destabilized earlier builds.

→ launchd + exponential backoff
Documentation

Full reference

The project is documented across focused, single-purpose files. SYSTEM_REFERENCE.md is the canonical source for architecture, component boundaries, and the complete script catalog.

SYSTEM_REFERENCE.md
Authoritative architecture, dependency boundaries, script inventory
RUNBOOK.md
Implementation path, failures, fixes, operational model
PRODUCT_OVERVIEW.md
Product surface, operating model, release vision
ADHOC_OTA.md
OTA install flow over Tailscale
RELEASE_QUEUE.md
TestFlight, tester management, preflight roadmap
Stack

What it's built with

Swift + SwiftUI
URLSessionWebSocketTask
JSON-RPC 2.0
Node.js
Codex app-server
Tailscale
xcodebuild
xcrun devicectl
tailscale serve
launchd
itms-services://
asc (App Store Connect)
What's Next

The phone becomes a
release pipeline

Mac-side TestFlight publish script. One-tap deploy from phone. Tester add/invite flows. App Review preflight scanner. Full release dashboard.

Follow the build → Watch the demo ↑