What to look for when choosing a cloud simulator

Quick answer

To choose a cloud simulator, check eight things: who operates it, how the build gets onto it, start time and how many sessions run at once, the isolation model, the evidence it returns, platform coverage, how you pay, and how it fits your CI and agents. If a coding agent is the main operator, weight operator, start time, isolation, evidence, and billing most.

An agent opened a pull request at 11:40 last night. Four hundred lines changed across the checkout flow. The description says "tested locally and verified the fix." You read the diff. It looks plausible. You have no idea whether the fix works on a live build.

That gap between code that reads correctly and an app that runs correctly is why simulators became such an important tool in an engineer's tool belt. When a coding agent writes a fix, someone or something has to run the app and look. When five agents run in parallel, that is five simulators. When the fix touches iOS and Android, that is even more.

So the question "which simulator should we use" has become a real infrastructure decision.

By the end of this post you will have a checklist of eight things to evaluate, and you will know which ones matter most if your main user is a coding agent rather than a person.

What a simulator is, and what it is not

A simulator runs a mobile operating system on a desktop or server so you can run an app without a physical phone. On iOS, Simulator ships with Xcode and runs an x86 or arm64 build of your app against a simulated iOS runtime. On Android, the Android Emulator ships with Android Studio and runs a full virtual device image, including the ARM or x86 system image.

The naming is loose in practice. Apple calls theirs a simulator because it does not emulate hardware; your app is compiled for the host CPU. Google calls theirs an emulator because it does. Most people say "simulator" for both, and this post does the same.

A simulator is not a real device. It will not tell you how your app behaves under thermal throttling, on a bad cell connection, or with a specific camera module. It is good at everything else: layout, navigation, state, crashes, most permissions flows, and the question "does the thing the agent says it fixed actually work."

A cloud simulator is the same runtime, hosted on a server somebody else operates, that you reach over the network. That is the category that has grown in the last year, and it is where most of the eight items below come from.

1. Who is going to operate it

Start here, because the answer changes every other item on the list.

If the primary operator is a person, you care about latency, frame rate, and how natural it feels to tap and scroll. A stutter is annoying. A 300 millisecond delay between tap and response is unusable.

If the primary operator is a coding agent, none of that matters much. The agent does not watch a video stream. It reads an accessibility snapshot, acts on an element reference, and checks the result. Tools like agent-device work this way: the agent gets a compact tree of what is on screen, presses @e2, and asks for a screenshot as evidence.

For an agent, the things that matter are: can it start a session from a command or an API call, does it run headless, does it return structured output, and does it produce artifacts (screenshots and recordings) that a human can inspect later. A simulator that requires a person to click "boot" in a GUI is a dead end for agent workloads.

Most teams need both. The agent verifies, then hands the session to a person who wants to poke at it. Check whether a simulator supports that handoff, or whether "agent mode" and "human mode" are two separate products.

2. How the build gets onto it

Every simulator session begins with installing an app binary. How that binary arrives is a bigger deal than it sounds.

With a local simulator, the binary is already on your machine because you just compiled it. With a cloud simulator, you have to get a .app, .ipa, or .apk from wherever it was built to wherever the simulator runs. The common path is: CI builds it, uploads it to storage, a script downloads it, and the simulator installs it. Each hop is a place to lose ten minutes and a place for the wrong build to end up in front of the reviewer.

The question to ask a vendor is: if my CI already produced the build, how many steps sit between "build finished" and "app is open on a simulator"? The best answer is zero, because the simulator service and the build service share storage and the simulator can install straight from a build URL. The worst answer involves a manual upload form.

This also decides whether an agent can close the loop on its own. If installing a build requires a person to drag a file, the agent has to stop and wait.

3. Start time and how many you can run at once

Two numbers define whether a simulator fits an automated workflow: how long it takes to get from "I need a simulator" to "the app is open," and how many you can have open at the same time.

Local simulators are fast to start once booted and slow to start cold. Xcode's Simulator takes tens of seconds to boot a fresh device and a good deal longer if it needs to download a runtime. Running four at once on a laptop is possible. Running four at once while also running your editor, your agent, and a video call is where the fans come on.

Cloud simulators trade a network round trip for a pool of machines. The vendor question here is whether sessions start cold (a VM boots when you ask) or warm (a VM is already waiting). Warm pools cost the vendor money to keep running, so they tend to be a paid-tier feature. Ask directly: what is the median time from request to a booted device, and does that change if I ask for ten at once?

On parallelism, ask what happens at the limit. A simulator service that queues your eleventh session with a visible position is workable. One that fails the request is going to page someone at 2 AM when an agent fleet hits the cap.

4. The isolation model

This item gets skipped in evaluations and it is the one that security teams will eventually ask about.

A simulator session has your unreleased app, whatever credentials that app holds, and whatever the agent driving it has been told. If a vendor runs multiple customers' simulators on one macOS machine as separate user accounts, your session shares a kernel with a stranger's. Sandbox escapes between macOS user accounts have a long history. This is not a theoretical concern.

The stronger model is one virtual machine per session. Each session boots in its own VM, runs, and is destroyed. Nothing persists between sessions unless you ask for it, and nothing from one customer can read memory or disk belonging to another. It is more expensive for the vendor, which is why not everyone does it.

Ask the plain question: is my session its own VM, or a process on a shared host? Follow up with what happens to the VM when the session ends, and whether access to a session follows the same permissions as the rest of your project or needs a separate login with its own access list to maintain.

5. What evidence comes out

An agent's claim "I verified this" is worth exactly as much as the evidence attached to it. The simulator is where that evidence gets made.

At minimum, a session should produce screenshots on demand. Better, it should produce a video recording of the whole session and a log of what the operator did: every tap, scroll, text entry, and wait, with timestamps. Best, it should produce a link that a reviewer can open and see all of that without installing anything.

The last one matters more than it seems. If the recording lives behind a login that only three people have, the reviewer on the PR will not open it. If it is a link that plays in a browser tab, they might. This is also how the cost of a simulator turns into something other than cost: every artifact a reviewer opens is a build nobody had to install and a device nobody had to register.

Check where artifacts are stored and how long they are retained. Then check whether they can be attached to a pull request automatically.

6. Platform coverage and how much fidelity you need

The obvious check: does it run iOS and Android? Less obvious: does it run both with the same feature set? Streaming an iOS Simulator to a browser and streaming an Android Emulator to a browser are different engineering problems, and some services ship one long before the other. If your reviewers need to see both, ask for a demo of both.

On fidelity, be honest about what the operator needs. A person doing exploratory QA wants something close to 60 frames per second and low latency. An agent needs the accessibility tree to be accurate and screenshots to be sharp. It does not care about frame rate at all. If your main workload is agent verification, paying for high-fidelity streaming is paying for something nobody watches.

Also check the device matrix. Can you pick an iPhone model and an iOS version, or run a tablet? Can you inject a location, a camera image, a slow network, or a low battery? Local tools like RocketSim and Radon have set a high bar for this on the desktop. Cloud services are catching up unevenly.

7. How you pay for it

There are three billing models in this category, and they favor different workloads.

Per parallel slot: you pay a flat monthly fee for the right to run N simulators at once, with unlimited minutes. This is the device-cloud norm. It is good for a test suite that runs constantly and bad for anything bursty, because you pay for the slot at 3 AM whether or not anything is running.

Per minute (or per second): you pay for time a session is actually running. This favors bursty workloads, agent sessions that last 45 seconds, and teams that do not yet know how much they will use. It punishes sustained load, where a flat fee ends up cheaper.

Dedicated hardware: you rent a whole machine. This is for compliance requirements or very high, steady volume.

Whatever the model, ask about spend controls before you sign. Agents get stuck in loops. A retry loop that boots a new simulator every 30 seconds all night is a real bill. You want per-project spend caps, alerts at thresholds, idle auto-suspend, and a way to kill a runaway session. If a vendor does not have those, the rest of the pricing conversation is beside the point.

8. How it fits the tools you already run

Finally, check whether the simulator behaves like infrastructure or like a destination.

Infrastructure means: your existing CI can start a session with a command, your existing agent (Claude Code, Codex, Cursor, or a homegrown one) can drive it with the tools it already knows, your existing build pipeline can feed it, and your existing project permissions govern who can open a session. You should not have to learn a new test framework or rewrite your agent prompts.

A destination means the vendor wants you to write tests in their format, drive devices through their dashboard, store results in their system, and train the team on all of it. That can be fine if you are starting from zero. It is a migration if you are not.

The tell is usually the CLI. If starting a session is one command with a platform flag, it is infrastructure. If it is a wizard, it is a destination.

The checklist

Score each item, then weight by who your operator is. A team whose simulators are mostly driven by agents should weight items 1, 3, 4, 5, and 7 heavily and care much less about 6.

Where to start

Do not evaluate on a demo app. Take one real pull request that an agent opened this week, and try to get from "PR exists" to "reviewer is watching the app run" on each candidate. Time it. Count the manual steps. That single exercise will answer most of the checklist faster than a vendor call.

Expo Simulators is built around items 1 through 5 on this list: sessions start from a command, install straight from a build, run on a remote device that belongs to your EAS project and is gone when the session stops, and produce artifacts an agent can attach to a PR. It works for any team and any app built for the iOS Simulator or the Android Emulator, including Flutter, React Native, and native apps. It fits the infrastructure you already have, and if you build with Expo, it installs the builds you already make with EAS Build. Expo Simulators is in early access, so read the current limitations before you plan around it.

Next in this series: when a cloud simulator is the right call and when the one on your laptop is still better.

Verified on 9 October 2026.