CLI reference (Coming Soon)

The CLI runs the exact Projects you build in AskUI Desktop. The same tests/, prompts/, plans/, and procedures/ drive both, author in the app, run anywhere with the CLI. See Install the CLI for setup and authentication.

askui run [<target>] [options]

Run from your Project root (or pass --project-root). Without arguments, askui run executes the whole tests/ folder against your local desktop.

What a run does

For every test, the agent works in phases: the folder's setup.md (if present) runs first, then the tests, then teardown.md, with any rules.md applied throughout. Each phase is one autonomous agent execution that reads the test file, drives the UI, and writes a report with screenshots. When you target a nested folder tree, setups run top-down, teardowns bottom-up, and rules.* accumulate per level; a failing setup blocks the tests it guards.

The target

Relative targets resolve against --project-root. Supported test file types: .txt, .md, .pdf, .csv, .json.

Plans

askui run --plan smoke

--plan <name> loads plans/<name> (any supported extension) and lets the agent select the matching tests before running them, e.g. a smoke plan that names a subset of suites. See How a Project is organized.

Run options

Surface options

# Drive a remote machine's AgentOS on its second display
askui run --controller-host 192.168.1.42 --display 2

# Run the suite against a browser
askui run --surface web

# Run against a specific Android device
askui run --surface android --device-serial emulator-5554

Model options

By default, inference runs through the AskUI hub, billed to your workspace and authenticated with ASKUI_WORKSPACE_ID/ASKUI_TOKEN, nothing to configure. The hub serves more than one model.

# Hosted models on the AskUI hub
askui run                                   # workspace default (Claude)
askui run --model-id gemini-3.5-flash       # hosted Gemini

# Bring your own model
askui run --model-provider anthropic --api-key $ANTHROPIC_API_KEY
askui run --model-provider openai-compatible \
  --endpoint-url http://localhost:8000/v1 \
  --model-id Qwen/Qwen3-VL-30B-A3B-Instruct

Credentials and .env

The CLI loads a .env file from the Project root at the start of every run, variables already set in the environment win. This carries the hub credentials (ASKUI_WORKSPACE_ID, ASKUI_TOKEN) and anything your own helper tools read.

Helper tools

The CLI extends the agent with your Project's custom tools: every *.dll in <project-root>/helpers/ is loaded, and each public tool class in it becomes available to the agent next to the built-in ones. A broken assembly never takes down the run, the problem is reported and the rest keeps loading.

What a run produces

Each run gets its own workspace folder (see --workspace), containing:

• Per-test artifacts: <test_name>/<test_name>_report.md with the result and step-by-step details, plus step_<n>.png screenshots.
• summary_report.md: one table over all executed tests with pass/fail/broken counts. Tests a plan selected but the run never reached appear as broken, so nothing goes missing silently.
• conversation.json: the full agent conversation, which AskUI Desktop re-renders as the conversation log.

The console shows each phase as it starts and finishes ([test] tests/login.md -> PASSED) and ends with the totals.

Exit codes

Running in CI

The exit codes make askui run CI-native: the job fails exactly when tests fail. A minimal GitHub Actions job:

jobs:
  ui-tests:
    runs-on: [self-hosted, askui]   # a runner with AgentOS installed
    env:
      ASKUI_WORKSPACE_ID: ${{ vars.ASKUI_WORKSPACE_ID }}
      ASKUI_TOKEN: ${{ secrets.ASKUI_TOKEN }}
    steps:
      - uses: actions/checkout@v4
      - run: askui run --plan smoke --workspace ./run-artifacts
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: askui-run
          path: ./run-artifacts

Pin --workspace to a fixed path so the artifact upload finds the reports, and upload with if: always() so failing runs keep their evidence. For setting up the machine under test, see AgentOS CI deployment.