CLI reference (Coming Soon)

The CLI runs the exact Projects you build in AskUI Desktop. The same tests/, plans/, procedures/, and utils/ drive both, author in the app, run anywhere with the CLI. See Install the CLI for setup and authentication.

askui run [<target>] [options]

Run from your Project root (or pass --project-root). Without arguments, askui run executes the whole tests folder against your local desktop (tests/, or the folder askui.json names as the tests path).

What a run does

For every test, the agent works in phases: the folder's setup.md (if present) runs first, then the tests, then teardown.md, with any rules.md applied throughout. Each phase is one autonomous agent execution that reads the test file, drives the UI, and writes a report with screenshots. When you target a nested folder tree, setups run top-down, teardowns bottom-up, and rules.* and ui.* accumulate per level, always starting at the tests root, so tests/rules.md and tests/ui.md apply to every run shape; a failing setup blocks the tests it guards.

The target

Relative targets resolve against --project-root. Supported test file types: .txt, .md, .pdf, .csv, .json.

Test plans

askui run --plan smoke

--plan <name> loads plans/<name>.yaml, a test plan, and runs exactly the tests it selects, in plan order. The plan's device list is ignored — the CLI runs on the surface its flags configure and prints a note saying so. A plan that selects no tests, or does not exist, is a configuration error (exit code 2).

Run options

Surface options

When the project's devices.json has exactly one profile matching the surface options, its name and prompt text join the agent's system prompt, the same device knowledge as an in-app run.

# Drive a remote machine's AgentOS on its second display
askui run --controller-host 192.168.1.42 --display 2

# Run the suite against a browser
askui run --surface web

# Run against a specific Android device
askui run --surface android --device-serial emulator-5554

Model options

By default, inference runs through the AskUI hub, billed to your workspace and authenticated with ASKUI_WORKSPACE_ID/ASKUI_TOKEN, nothing to configure. The hub serves more than one model.

# Hosted models on the AskUI hub
askui run                                   # workspace default (Claude)
askui run --model-id gemini-3.5-flash       # hosted Gemini

# Bring your own model
askui run --model-provider anthropic --api-key $ANTHROPIC_API_KEY
askui run --model-provider openai-compatible \
  --endpoint-url http://localhost:8000/v1 \
  --model-id Qwen/Qwen3-VL-30B-A3B-Instruct

Credentials and .env

The CLI loads a .env file from the Project root at the start of every run, variables already set in the environment win. This carries the hub credentials (ASKUI_WORKSPACE_ID, ASKUI_TOKEN) and anything your own helper tools read.

Helper tools

The CLI extends the agent with your Project's custom tools: every *.dll in <project-root>/helpers/ is loaded, and each public tool class in it becomes available to the agent next to the built-in ones. A broken assembly never takes down the run, the problem is reported and the rest keeps loading.

What a run produces

Each run gets its own workspace folder (see --workspace), containing:

• Per-test artifacts: <test_name>/<test_name>_report.md with the result and step-by-step details, plus step_<n>.png screenshots.
• summary_report.md: the run at a glance — totals with per-status counts and run duration, one table over all tests with status and duration each, and detail sections quoting every warned, failed, or broken test's issues. Tests a plan selected but the run never reached appear as broken, so nothing goes missing silently.
• conversation.json: the full agent conversation, which AskUI Desktop re-renders as the conversation log.

The console shows each phase as it starts and finishes ([test] tests/login.md -> PASSED) and ends with the totals.

Exit codes

Running in CI

The exit codes make askui run CI-native: the job fails exactly when tests fail. A minimal GitHub Actions job:

jobs:
  ui-tests:
    runs-on: [self-hosted, askui]   # a runner with AgentOS installed
    env:
      ASKUI_WORKSPACE_ID: ${{ vars.ASKUI_WORKSPACE_ID }}
      ASKUI_TOKEN: ${{ secrets.ASKUI_TOKEN }}
    steps:
      - uses: actions/checkout@v4
      - run: askui run --plan smoke --workspace ./run-artifacts
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: askui-run
          path: ./run-artifacts

Pin --workspace to a fixed path so the artifact upload finds the reports, and upload with if: always() so failing runs keep their evidence. For setting up the machine under test, see AgentOS CI deployment.