Cloud device lab: repeatable QA on cloud-Android, human-gated
A controlled cloud-Android lab for repeatable QA scenarios — prepared data, scripted runs, per-device logging and a human check at the end of every run.
Context
Testing a mobile app's onboarding at scale needs many clean devices and many repeatable runs — infrastructure a single phone on a desk cannot provide. Emulators drift from real device behavior, and reusing one physical device contaminates every run after the first: cached state, stale accounts, leftover permissions.
The lab answers that with cloud-Android devices treated as disposable, isolated fixtures. The work is legitimate QA by design, and the architecture keeps it that way: prepared datasets, safe synthetic identities that map to no real person, and a human at the end of every run instead of a silent automated pass.
Approach
The core design decision is that a test run is a pipeline artifact, not an interactive session. Every run passes through the same five stages — prepare the dataset, launch, execute the RPA scenario on a dedicated device, log each step, hand off to a human verifier — so any two runs of the same scenario are directly comparable.
Isolation is enforced at the device level: each run gets its own cloud-Android device through GeeLark, so no state leaks between scenarios and a failure in one run cannot poison the next. Identities used in onboarding flows are synthetic by construction — generated test data, never harvested or real.
How a run works
The scenario runner assembles the input dataset from text-dataset templates, provisions a dedicated cloud-Android device via the GeeLark API and starts the RPA scenario on it. Actions execute step by step — app install, onboarding screens, form input — with each step written to Supabase as it happens, tagged with the device and run identifiers.
Orchestration is hosted on Railway, so the control plane lives outside the devices it manages; an Android prototype app serves as the reference target for scenario development. When the scripted portion completes, the run does not auto-close: it lands in a verification queue where a human inspects the logged trail and the end state before the run is marked done.
Key engineering details
Repeatability comes from separating the three things that usually get tangled in mobile QA: the data (versioned text datasets), the behavior (RPA scenarios as scripts, not recordings of a hand on a screen) and the substrate (interchangeable cloud devices). Change any one and the other two hold still — which is what makes a diff between two runs meaningful.
The human-verification gate is structural, not procedural: a run's terminal state is unreachable without a manual check, so the guarantee does not depend on operator discipline. Per-device, per-step logging in Supabase means every claim a run makes about app behavior can be traced back to a timestamped action.
Outcome & what shipped
The infrastructure prototype covers the full contour: scenario runner, GeeLark device orchestration, Railway-hosted control plane, Supabase step logging and the human-verification gate — supporting legitimate mass QA of mobile apps, onboarding verification and reproducible RPA scenarios on a cloud farm of test devices.
Next on the roadmap: expanding the scenario library, run-to-run comparison across device batches, and hardening the verification gate with structured checklists per scenario type.
What we built
Scenario runner
Prepares the dataset, launches a run and executes RPA actions on a dedicated device.
Cloud-Android orchestration
Dedicated cloud devices provisioned and managed through GeeLark; one device per run.
Per-device logging
Every step written to Supabase with device and run identifiers for later inspection.
Human verification
Each run terminates in a manual check; identities are synthetic and safe by construction.