Skip to main content
Your coding agent can take the phone out of your hands. It opens your app on a real device, sees what’s on screen, taps and types its way through, and tells you what happened — you stay in your editor. This page is what it can do there. How agents connect to noqa is on CLI & MCP; setup and every option live in the CLI reference.

​
Connect a device

One device is active at a time, and every screen read and action goes to it.

​
Read the screen

Your agent reads the screen on every step, so how it reads drives the token bill. The three ways differ enormously — measured on the same screen (the Lingua Talks paywall):
  • noqa screen --full is the raw accessibility tree straight from Appium — every wrapper, container, and duplicated label. Complete, but noisy and heavy, and it balloons on complex screens.
  • A screenshot is cheap in tokens, but it’s pixels only: the model has to infer where elements are and read small text visually, which hurts tap precision and misses anything off-screen.
  • noqa screen is our cleaned element tree — the redundant nodes stripped, the meaningful elements kept with their exact bounding boxes. Here that’s ~7× fewer tokens than the raw tree, and lighter than a screenshot — while still giving precise, structured positions to act on.

​
Act on what you read

noqa screen already hands you every element’s box, so acting is usually just arithmetic: take the centre of the target’s relative_bbox_2d and tap it. Coordinates are relative (0–1000 on both axes), so they hold across devices and resolutions.

​
When the tree runs out

Plenty of screens have no usable tree at all. A Unity or Unreal game draws its own UI; a canvas, a WebView or an ad overlay comes back as one flat box with nothing inside. Sometimes the tree is there but lies — the coordinates look right and the tap changes nothing. That is what grounding is for: you describe the target in plain language and noqa finds it on the screen itself, placing the touch for you. It works where locators don’t, because it never asks the app for an element tree — a Unity button, a WebView, an ad overlay and a native control are all just things on the screen.
Describe what a person would look for — position, colour, and the words on it all help (“the X in the top-right corner of the paywall”). Grounding is free within a daily quota; past that it draws on credits. Reach for it as soon as the tree stops being a reliable source of positions: noqa screen times out or takes a long time to answer, comes back empty or with far fewer elements than the screen clearly has, carries no texts or labels — or your tap on its coordinates did nothing.

​
Control the app and the system

restart-app is how your agent gets back to a known state without touching the device by hand.

​
The loop

Put together, it’s inspect → act → verify, one step at a time and never an action fired blind:
  1. Inspect — noqa screen to confirm the target is there and the UI is in the expected state.
  2. Act — one noqa action …, on the coordinates you just read; by description when the tree gave you none.
  3. Verify — noqa screen again, or noqa screenshot for a visual check.
That loop is enough to check a feature you just built. To turn it into saved test cases that run on every release, see Generate test cases with AI.

Generate test cases with AI

Your agent writes the cases and runs them

CLI reference

Every command and option