Test plans

A test plan is a saved selection of tests and the device they run on. You build it once by checking tests and picking a device, then run the whole selection with one click, again and again, or on a schedule. Nothing infers what runs: a plan contains exactly the tests you checked.

Plans are how you turn a folder of tests into the sets your team actually runs: a fast smoke set for every change, a full regression pack for a release, a nightly suite.

Build a plan

You assemble a plan on the Test Plans page. There is no file to write by hand: you check tests, pick a device, and the plan is saved for you.

1. Open Test Plans and click New test plan.
2. Name the plan, for example smoke.
3. Check the tests to include. Check individual files, check a folder to include everything inside it, or check All tests for the whole project. A file covered by a checked folder shows as included; uncheck the folder to pick files one by one.
4. Pick the device the plan runs on. Every profile from the Devices page is offered, connected or not: a device only has to be reachable when the plan runs, not while you build it.
5. Click Create plan.

A checked folder stays live: a test added to that folder later is part of the plan automatically, with nothing to re-check. The Test Plans page lists each plan with its test count and device, and the pencil on a row reopens it for editing.

Run it with one click

Press Run on the plan. Every selected test runs on the plan's device, in the order listed, with no further input. The run appears on the Runs page, and another run on the same device queues behind it.

Run is disabled, with the reason in its tooltip, while a plan selects no tests, has no device, or points at a device that no longer exists. If the device is busy or two runs are already in flight, a dialog offers to queue the run.

A plan run accounts for every test it selected:

• Selected but not reached: a test the run never got to is reported as BROKEN in the run report, so nothing is dropped silently.
• Renamed or deleted: a plan entry whose file is gone is skipped with a notice in the conversation log, and the rest of the run continues.

You can also run a plan without the app, with askui run --plan smoke. The CLI runs the selection on the surface its own flags configure and ignores the plan's device.

Put it on a schedule

Open the plan on Test Plans and use its schedule action to run it unattended, daily at a time or every few minutes. A scheduled plan runs on the device saved in the plan, so there is no device to pick. See Scheduling for cadence and management.

Share it through Git

A plan is a plain file in your project's plans/ folder, versioned and reviewed like the rest of your testware. Commit it and your team runs the same selection; change it and the change shows up in review as a small, readable diff. The device is stored by its profile id, not its name, so renaming a device never touches your plan files.

If the plan's device profile is later deleted, the plan's row flags it and the editor shows the orphaned device with a remove button: pick a new device and the plan runs again.

Under the hood: the plan file

You never edit it directly, but a plan is just yaml, which is what lets it diff so cleanly:

version: 1
tests:
  - tests/login/
  - tests/checkout/happy_path.md
devices:
  - id: a1b2c3

• tests: your selection, in run order. An entry ending in / is a whole folder, expanded when the run starts (this is why a checked folder keeps up with tests added to it later).
• devices: the device the plan runs on, by profile id from devices.json. The file holds a list for forward compatibility; the app sets exactly one.

The plans/ folder is created with your first plan.

Older prose plans

Plans used to be free-form Markdown files that an agent interpreted at run time. Those files are no longer runnable and no longer appear on the Test Plans page; they stay on disk until you delete them. Recreate the selection as a test plan and the checkboxes cover what the prose described.