Test plans

A test plan is a saved selection of tests and the device they run on. You build it once by checking tests and picking a device, then run the whole selection with one click, again and again, or on a schedule. Nothing infers what runs: a plan contains exactly the tests you checked.

Plans are how you turn a folder of tests into the sets your team actually runs: a fast smoke set for every change, a full regression pack for a release, a nightly suite.

Build a plan

You assemble a plan on the Test Plans page. There is no file to write by hand: you check tests, pick a device, and the plan is saved for you.

1. Open Test Plans and click New test plan.
2. Name the plan, for example smoke.
3. Check the tests to include. Check individual files, check a folder to include everything inside it, or check All tests for the whole project. A file covered by a checked folder shows as included; uncheck the folder to pick files one by one.
4. Pick the device the plan runs on. Every profile from the Devices page is offered, connected or not: a device only has to be reachable when the plan runs, not while you build it.
5. Click Create plan.

A checked folder stays live: a test added to that folder later is part of the plan automatically, with nothing to re-check. The Test Plans page lists each plan with its test count and device, and the pencil on a row reopens it for editing.

Order the tests

Tests run top to bottom in the order shown in the plan's tree, and you can change that order by dragging. Hover a selected row, grab its drag handle, and drop it above or below one of its siblings.

• A folder is one ordered slot: drag the folder to move everything under it at once. To place a test ahead of tests in another folder, move the folder.
• Inside a whole-checked folder you can still order the tests: expand it and drag its rows. The order you set is pinned, and the folder keeps auto-including tests you add later (those run after the ones you pinned). Until you drag inside it, a checked folder runs its tests alphabetically.

Selecting a test never changes the order — only dragging does. The order is saved in the plan file, so it travels with your project.

Run it with one click

Press Run on the plan. Every selected test runs on the plan's device, in the order listed, with no further input. The run appears on the Runs page, and another run on the same device queues behind it.

Run is disabled, with the reason in its tooltip, while a plan selects no tests, has no device, or points at a device that no longer exists. If the device is busy or two runs are already in flight, a dialog offers to queue the run.

A plan run accounts for every test it selected:

• Selected but not reached: a test the run never got to is reported as BROKEN in the run report, so nothing is dropped silently.
• Renamed or deleted: a plan entry whose file is gone is skipped with a notice in the conversation log, and the rest of the run continues.

You can also run a plan without the app, with askui run --plan smoke. The CLI runs the selection on the surface its own flags configure and ignores the plan's device.

Put it on a schedule

Open the plan on Test Plans and use its schedule action to run it unattended, daily at a time or every few minutes. A scheduled plan runs on the device saved in the plan, so there is no device to pick. See Scheduling for cadence and management.

Share it through Git

A plan is a plain file in your project's plans/ folder, versioned and reviewed like the rest of your testware. Commit it and your team runs the same selection; change it and the change shows up in review as a small, readable diff. The device is stored by its profile id, not its name, so renaming a device never touches your plan files.

If the plan's device profile is later deleted, the plan's row flags it and the editor shows the orphaned device with a remove button: pick a new device and the plan runs again.

Under the hood: the plan file

You never edit it directly, but a plan is just yaml, which is what lets it diff so cleanly:

version: 1
tests:
  - tests/login/
  - tests/checkout/happy_path.md
devices:
  - id: a1b2c3

• tests: your selection, in run order. An entry ending in / is a whole folder, expanded when the run starts (this is why a checked folder keeps up with tests added to it later). When you order tests inside a checked folder, those files are listed explicitly before the folder entry: they run in that order first, then the folder contributes anything not already listed.
• devices: the device the plan runs on, by profile id from devices.json. The file holds a list for forward compatibility; the app sets exactly one.

The plans/ folder is created with your first plan.

Older prose plans

Plans used to be free-form Markdown files that an agent interpreted at run time. Those files are no longer runnable and no longer appear on the Test Plans page; they stay on disk until you delete them. Recreate the selection as a test plan and the checkboxes cover what the prose described.