Test plans

A test plan is your test selection: which cases run together, and on which device. The classic examples, a fast smoke set for every change, the full regression pack for the release, a nightly suite.

A plan is a yaml file in plans/. You never write it by hand — you check tests and pick a device on the Test Plans page and the file is written for you — but it reads (and diffs) like this:

version: 1
tests:
  - tests/login/
  - tests/checkout/happy_path.md
devices:
  - id: a1b2c3

• tests: the selection, in execution order. An entry ending in / is a whole folder — it is expanded when the run starts, so a test added to the folder later is included without touching the plan.
• devices: the device the plan runs on, by profile id from devices.json (a list in the file, one entry via the app). Only the id is stored — the app shows the device's current name, so renaming a device never touches your plan files.

The run contains exactly what the plan lists. Nothing infers the selection, and a plan that selects no tests cannot be run.

Create a plan

1. Open Test Plans and click New test plan. A dialog opens.
2. Type a name, for example smoke.
3. Check tests in the tree: individual files, a folder to include everything in it, or All tests to run the whole project. Files inside a checked folder show as included; uncheck the folder to pick them individually.
4. Pick the device the plan targets. Every profile from the Devices page is offered — the device does not need to be connected while you edit, only when the plan runs.
5. Click Create plan. The plan is saved as plans/smoke.yaml (the folder is created with the first plan).

The page lists each plan with its test count and device; the pencil on a row reopens the dialog. A plan change shows up in review as a small yaml diff, like the rest of your testware.

Run a plan

Press Run on the plan. A run starts on the plan's device and executes every selected test — no further input. It appears on the Runs page; runs on the same device queue behind each other.

The button is disabled while the plan selects no tests or no device, with the reason in the tooltip. If the plan's device is busy (or the two-runs limit is reached), a dialog asks whether to queue the run.

Two things to know about how a plan run reports:

• A selected test the run never reaches is listed as BROKEN in the run report — selected means accounted for.
• A plan entry whose file was renamed or deleted is skipped with a notice in the conversation log; the run continues with the rest.

You can also run a plan headless with askui run --plan smoke — the CLI runs the selection on the device its flags configure and ignores the plan's device list — or on a schedule, which runs it on the plan's device the same way.

If a device disappears

A plan remembers its device by profile id. When the referenced profile no longer exists, the plan's row says so and the edit dialog shows the orphaned id with a remove button — pick a new device and the plan runs again.

Older prose plans

Plans used to be free-form Markdown files that an agent interpreted at run time. Those files are no longer runnable and no longer appear on the Test Plans page (they stay on disk until you delete them). Recreate the selection as a test plan — the checkboxes cover what the prose described.