Conversation 4 acts · 6 tool calls ▾ Autofollow

Quiet Detailed Raw

{/* Act 1, Setup */}

Setup · setup.md Pass

Ensure the desktop is on the home screen.

{/* Act 2, Test */}

Test · tests/hello_world.md Pass 5 calls

Open the Notes app and type "Hello World".

Expected The note shows "Hello World".

I'll open Notes from the Start menu, create a note, and type the text.

📷 screenshot → captured 120ms

🖱️ click 134, 174 → clicked 140ms

🖱️ click 460, 27 → clicked 98ms

⌨️ type Hello World → typed 210ms

📷 screenshot → captured

Notepad showing the text Hello World, captured by the agent

The note shows "Hello World", step verified.

{/* Act 3, Teardown */}

Teardown · teardown.md Pass

Close the Notes app without saving.

🖱️ click 1884, 21 → clicked 88ms

{/* Act 4, Upcoming (predicted next step) */}

Upcoming · tests/save_note.md Pending

Next: save the note and reopen it to confirm the text persists.

```json [ { "role": "user", "content": { "type": "act", "kind": "setup", "file": "setup.md" } }, { "role": "user", "content": { "type": "act", "kind": "test", "file": "tests/hello_world.md" } }, { "role": "assistant", "content": { "type": "text", "text": "I'll open Notes, create a note, and type the text." } }, { "role": "assistant", "content": { "type": "tool_use", "name": "click", "input": { "x": 134, "y": 174 } } }, { "role": "user", "content": { "type": "tool_result", "content": "clicked" } }, { "role": "assistant", "content": { "type": "tool_use", "name": "type", "input": { "text": "Hello World" } } }, { "role": "user", "content": { "type": "tool_result", "content": "typed" } }, { "role": "assistant", "content": { "type": "text", "text": "The note shows 'Hello World' — step verified." } }, { "role": "user", "content": { "type": "act", "kind": "teardown", "file": "teardown.md" } }, { "role": "assistant", "content": { "type": "tool_use", "name": "click", "input": { "x": 1884, "y": 21 } } }, { "role": "user", "content": { "type": "tool_result", "content": "clicked" } } ] ``` Follow it top to bottom, like watching over a colleague's shoulder: The run starts with **① Setup, ② Test, ③ Teardown**, the same three phases you know from any test run, each a numbered block with a **Pass / Fail** badge. In our run, act ① put the desktop into the starting state, then act ② picked up `hello_world.md`. Inside act ②, the agent first **tells you its plan** in plain language, *"I'll open Notes from the Start menu, create a note, and type the text."* These text lines are the agent thinking out loud; read them like a colleague narrating their work. Then it **acts on the screen**, the smaller rows, its *tool calls*. It takes a **screenshot** to see the desktop, **clicks** at position `134, 174` (where it spotted the Notes icon, the agent works in screen coordinates, like a human aiming the mouse), clicks again for a new note, and **types** `Hello World`. Every tool call shows its result (`→ clicked`); a failed one turns red with the error. Then it **checks its work**: a final screenshot, expanded so you see exactly what the agent saw, and its conclusion, *"The note shows Hello World, step verified."* That screenshot is your proof the pass is real. Act ③ cleans up (closing Notes), and the dashed **④ Upcoming** row at the bottom is the agent **predicting its next step**, during a live run you always see where it's heading before it gets there. If it's too much detail, switch from **Detailed** to **Quiet** for just the phases; **Raw** shows the underlying data. **Autofollow** keeps the view pinned to what the agent is doing right now. ### Test report [#test-report] Where the conversation log is the **test execution log**, the report is the **test execution report**: the structured record of one test case, its result, the evidence behind it, and any deviations. It follows the shape you know from manual test documentation. Here's the report for the `hello_world` run:

Test Case Report: hello_world

Test Case ID: hello_world
Date: 2026-07-17
Status: PASSED

Summary

The agent opened the Notes app, created a new note, and typed "Hello World". All steps passed; one incident was resolved during execution (see Issues).

Preconditions

1\. The desktop is on the home screen.

Test Steps

1. Open the Notes app , Status: PASSED

Agent Interpretation: Located the Notes icon on the taskbar and clicked it.
Expected: The Notes app opens.
Actual: Notes opened; a "What's new" popup appeared and was dismissed.

2. Create a new note , Status: PASSED

Agent Interpretation: Clicked the new-tab button to start an empty note.
Expected: An empty note is focused.
Actual: A new empty tab opened with the cursor in the text area.

3. Type "Hello World" , Status: PASSED

Agent Interpretation: Typed the text and took a screenshot to verify.
Expected: The note shows "Hello World".
Actual: The note shows "Hello World".

Postconditions

1\. The note shows the text "Hello World".

Issues

1\. A "What's new" popup appeared after launching Notes and blocked the text area. Dismissed it and continued. Consider whether this popup should appear on every launch.

Conclusion

All steps passed. Creating and writing a note works as expected; the launch popup was the only deviation.

```markdown # Test Case Report: hello_world **Test Case ID:** hello_world **Date:** 2026-07-17 **Status:** PASSED ## Summary The agent opened the Notes app, created a new note, and typed "Hello World". All steps passed; one incident was resolved during execution (see Issues). ## Preconditions 1. The desktop is on the home screen. ## Test Steps 1. **Open the Notes app** — Status: **PASSED** - **Agent Interpretation:** Located the Notes icon on the taskbar and clicked it. - **Expected:** The Notes app opens. - **Actual:** Notes opened; a "What's new" popup appeared and was dismissed. 2. **Create a new note** — Status: **PASSED** - **Agent Interpretation:** Clicked the new-tab button to start an empty note. - **Expected:** An empty note is focused. - **Actual:** A new empty tab opened with the cursor in the text area. 3. **Type "Hello World"** — Status: **PASSED** - **Agent Interpretation:** Typed the text and took a screenshot to verify. - **Expected:** The note shows "Hello World". - **Actual:** The note shows "Hello World". - ![Step 3](step_3.png) ## Postconditions 1. The note shows the text "Hello World". ## Issues 1. A "What's new" popup appeared after launching Notes and blocked the text area. Dismissed it and continued. Consider whether this popup should appear on every launch. ## Conclusion All steps passed. Creating and writing a note works as expected; the launch popup was the only deviation. ``` Section by section: **Header:** Test case ID, execution date, overall status, your at-a-glance answer.\ **Summary:** The result in one or two sentences. Start here.\ **Preconditions:** The entry criteria the agent verified before executing, rules out an environment problem in disguise.\ **Test Steps:** Per step: status, **expected vs. actual result**, screenshot as evidence. This is where you localize which step diverged.\ **Postconditions:** The exit state the test asserted.\ **Issues:** Your incident log, deviations the agent hit during execution, including ones it resolved itself, like the "What's new" popup above, an unexpected dialog, or leftover data it had to delete first. A test can be `PASSED` and still list issues, and they often point at real findings a green checkmark would hide, review this section even when everything is green.\ **Conclusion:** The overall result and its justification. Five statuses exist, for steps and for the test overall, the test takes the worst status of its steps: **`PASSED`**: the actual result matched the expected one. The agent never bends a test to reach a pass.\ **`WARN`**: passed, but with a caveat worth your attention (slow response, minor visual deviation).\ **`FAILED`**: the step executed, but the actual result didn't match the expected one.\ **`SKIPPED`**: intentionally not executed, e.g. a precondition wasn't met.\ **`BROKEN`**: couldn't execute at all: a crash or an infrastructure problem, not a functional result. For details and the cross-run summary report, see [Reading a run report](/docs/results/run-report). And when the log or report shows the agent got something wrong, clicked the wrong element, misread a screen, drew the wrong conclusion, you don't debug code, you **teach it**: sharpen the wording of the test case, add a rule, or give it the missing knowledge in the prompts. How test cases, rules, and prompts work together is covered in [Prompting best practices](/docs/best-practices/prompting-best-practices). {/* TODO: retarget this link to the Concepts section once it exists. */} Once the example passes, open `tests/ui.md` and describe your own app, its main screens, how to tell you're signed in, and any quirks. It's the single most important file for reliable runs. See [Prompting best practices](/docs/best-practices/prompting-best-practices). # Install AskUI Desktop (/docs/get-started/install-desktop) The entry point for authoring and running AI-driven UI tests. Runs on Windows and macOS. **Download the installer** for your platform: Also on your [AskUI Hub](https://hub.askui.com) account and the [Releases page](https://github.com/askui/integrated-task-platform/releases/latest) (always newest). **Run the installer.** Run the `.exe` and accept the prompts. Open the `.dmg` and drag AskUI Desktop to Applications. **Launch and sign in** with your AskUI account. On first launch the onboarding wizard sets up permissions and your first project. **Run [your first test](/docs/get-started/first-run).** # License key (/docs/get-started/licensing) There are two ways to run AskUI Desktop: | Mode | How you get in | Where inference runs | | ----------------- | ---------------------------------------------------------------------------- | ---------------------------------------------------------- | | **AskUI account** | [Sign in](/docs/get-started/install-desktop) with your AskUI account (Auth0) | AskUI hub, billed to your workspace | | **License key** | Activate a local license key, no account needed | **Your own model provider** (you add it during activation) | License mode is for teams that run against their **own** Anthropic or OpenAI‑compatible endpoint and don't want inference to route through the AskUI hub, air‑gapped setups, self‑hosted models, or an existing model contract. Because License mode never talks to the AskUI hub for inference, it **requires** a model provider. Activation walks you straight into [connecting one](/docs/extending/model-providers). ## Activate [#activate] **Open AskUI Desktop.** On the welcome screen choose **Activate with a license key** instead of signing in. **Paste your license key** and select **Continue**. **Add your model provider.** License mode runs on your own model, so the next step is the [model provider form](/docs/extending/model-providers). Enter your endpoint and key, then **Finish & open AskUI**. You're now in the app. AskUI Desktop remembers the license and model provider, so subsequent launches open straight into your projects. ## Managing your license [#managing-your-license] * Your license and its plan tier are shown in the **profile menu** (top‑right). * To switch machines or hand the seat back, **remove the license key** from the profile menu. Removing it returns you to the welcome screen. * If you later sign in with an AskUI account, the account takes precedence over a stored license key. Not sure which mode you want? If you don't already operate your own model endpoint, [sign in with an AskUI account](/docs/get-started/install-desktop) instead, it's the zero‑config path. ## Next [#next] # Uninstall AskUI (/docs/get-started/uninstall) Open **Settings** and go to **Apps** (or open **Control Panel** and go to **Programs and Features**). Find **AskUI Desktop** in the list of installed applications. Select it and click **Uninstall**, then follow the prompts. The uninstaller removes the application, its Start-menu and desktop shortcuts, and its registry entries. Your settings and sign-in data under `%LOCALAPPDATA%\AskUIDesktop` stay on disk, delete that folder too for a complete removal. **Silent uninstall:** ```powershell & "$env:LOCALAPPDATA\Programs\AskUI Desktop\uninstall.exe" /S ``` Open **Finder** and go to **Applications**. Drag **AskUI Desktop** to the Trash, or right-click and select **Move to Trash**. Empty the Trash to complete the removal. # AskUI in CI (coming soon) (/docs/guides/askui-in-ci) CI execution needs the AskUI CLI, which is not publicly available yet. This page describes the setup you will use once it ships, so you can plan for it. The project your team authors in AskUI Desktop will run unchanged on a build server, same testware, headless execution. ## 1: Get the CLI onto the runner [#1-get-the-cli-onto-the-runner] [Install the CLI](/docs/running-tests/install-cli) and provide credentials as environment variables: `ASKUI_WORKSPACE_ID` and `ASKUI_TOKEN` (create the token in the [Hub](/docs/account-billing/workspaces)). ## 2: Give it a device [#2-give-it-a-device] A [headless browser](/docs/devices/web-browser) runs on the bare runner. Desktop targets need a test machine with [AgentOS](/docs/agentos/deployment/ci) the runner can reach. ## 3: Run a plan [#3-run-a-plan] ```bash askui run --plan smoke --project-root . ``` Gate the pipeline on the exit code, and archive `agent_workspace/` as the build artifact, reports and screenshots included; the [CLI reference](/docs/running-tests/cli) shows the report-path and CI options. Which team runs what in CI, and who owns the results, is a working-model question: [Sharing the project](/docs/concepts/team-setups). # Test a web app (/docs/guides/test-a-web-app) The same journey as [Android](/docs/guides/test-an-android-app), for the web. ## 1: Define the browser [#1-define-the-browser] [Create a Web browser profile](/docs/devices/web-browser): pick Chromium, Firefox, or WebKit and set the **Start URL** to your application, one profile per environment (`Staging web app`, `Prod smoke`) keeps the same tests reusable across them. ## 2: Connect [#2-connect] **Connect** on the profile card, the first connect downloads the browser bundle once. No AgentOS, no further setup. ## 3: Write the test case [#3-write-the-test-case] [Create the case](/docs/writing-tests) and name the browser profile in it. Web-specific craft: describe elements as the user sees them ("the blue Checkout button in the cart"), state the expected page or dialog after every navigation, and put login into a [procedure](/docs/writing-tests/plans) with the credentials as [secrets](/docs/extending/secrets). ## 4: Run and analyze [#4-run-and-analyze] **Run on** the browser profile, headless keeps your screen free, a visible window lets you watch alongside the [live conversation](/docs/running-tests#watch-it-live). Failures land in the [run report](/docs/results/run-report) with a screenshot per step. # Test an Android app (/docs/guides/test-an-android-app) The whole journey for one goal: a passing (or usefully failing) test against an Android app. Each step links to the feature page with the full detail. ## 1: Define the device [#1-define-the-device] Emulator or real phone: * **Emulator**: [create an Android emulator profile](/docs/devices/android-emulator): device type, Android version, lifecycle. No phone needed; missing SDK tooling is a one-click **Set up**. * **Real phone**: enable USB debugging, plug it in, **Add as profile** from the discovered list: [Android phone](/docs/devices/android-phone). ## 2: Start it [#2-start-it] **Start** on the profile card. First emulator boot takes minutes, the [card walks through the states](/docs/devices/android-emulator#run-it) (Set up → Creating → Booting → Ready). Green dot = the device is an available test environment. ## 3: Write the test case [#3-write-the-test-case] [Create the case](/docs/writing-tests) under `tests/` and name the target device in it. Android-specific craft: * Name apps **as the user sees them** ("open the Calculator app"), not by package id, the agent works the launcher like a person. * **Verify the app is installed first.** Emulator images are lean; a stock Google-APIs image ships without Calculator, for example. Either state it as a precondition or pick an app the image has (Settings, Chrome, Clock). * Steps like swipes and the app drawer work in plain language: "open the app drawer", "swipe down to open notifications". ```md title="tests/open_clock_android.md" # Open the Clock app Run this on the **Pixel 9** device. ## Preconditions - The device shows the home screen. ## Steps 1. Open the app drawer. 2. Launch the **Clock** app. ## Postconditions - Test passes if the Clock app is open and shows the time. ``` ## 4: Run it and watch [#4-run-it-and-watch] Pick the Android profile in **Run on**, press **Run**, and open the **Runs** page, [the conversation streams live](/docs/running-tests#watch-it-live): every decision, tap, and device screenshot as it happens. ## 5: Analyze the result [#5-analyze-the-result] The [run report](/docs/results/run-report) gives per-step expected vs. actual with a screenshot each, your incident log when something failed. The [conversation log](/docs/results#conversation-log) holds the complete execution record. Remember the agent may have taken a different path than you imagined, [judge the result, not the path](/docs/best-practices/writing-good-tests#non-deterministic-execution). # Legacy SDKs (/docs/legacy) The Python SDK and TypeScript SDK are legacy products. They receive no bug fixes, security updates, or new features. We strongly recommend migrating to **AskUI Desktop** and the **AskUI CLI**. AskUI previously offered two code-based SDKs for authoring UI automation: * **Python SDK**: write automation scripts in Python against the AskUI runtime. * **TypeScript SDK**: write automation scripts in TypeScript/Node.js against the AskUI runtime. Both SDKs required writing code to interact with the AskUI agent. They are no longer actively developed or supported. ## Why we moved on [#why-we-moved-on] The SDK approach required users to learn an API, manage a language runtime, and maintain code as their UI changed. AskUI Desktop replaces all of that with plain-language prompts that describe what to do rather than how to do it, no selectors, no boilerplate, no SDK version upgrades. ## What to use instead [#what-to-use-instead] Author tests in plain Markdown and run them interactively on your machine. No code, no selectors, no SDK. Run the same projects headless in CI or on a schedule with a single command. Ships with the Desktop App installer. Get from zero to a passing test in under 10 minutes. # Python SDK (legacy) (/docs/legacy/python-sdk) The Python SDK is a legacy product. It receives no bug fixes, security updates, or new features. Use **AskUI Desktop** and the **AskUI CLI** for all new projects. The AskUI Python SDK allowed writing UI automation scripts in Python against the AskUI agent runtime. It is no longer actively developed or supported. ## Migrate to AskUI Desktop [#migrate-to-askui-desktop] AskUI Desktop replaces the SDK workflow entirely. Instead of writing Python code to interact with UI elements, you describe test steps in plain Markdown and the AI agent carries them out on the real screen. You do not need to rewrite automation logic line by line. The prompt-based approach means most test scenarios can be expressed more concisely than the equivalent SDK code. **[Get started with AskUI Desktop](/docs/get-started/install-desktop)** # TypeScript SDK (legacy) (/docs/legacy/typescript-sdk) The TypeScript SDK is a legacy product. It receives no bug fixes, security updates, or new features. Use **AskUI Desktop** and the **AskUI CLI** for all new projects. The AskUI TypeScript SDK allowed writing UI automation scripts in TypeScript/Node.js against the AskUI agent runtime. It is no longer actively developed or supported. ## Migrate to AskUI Desktop [#migrate-to-askui-desktop] AskUI Desktop replaces the SDK workflow entirely. Instead of writing TypeScript code to interact with UI elements, you describe test steps in plain Markdown and the AI agent carries them out on the real screen. You do not need to rewrite automation logic line by line. The prompt-based approach means most test scenarios can be expressed more concisely than the equivalent SDK code. **[Get started with AskUI Desktop](/docs/get-started/install-desktop)** # Environment variables (/docs/reference/environment-variables) ## Authentication [#authentication] | Variable | Description | | -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- | | `ASKUI_WORKSPACE_ID` | Your workspace identifier. Required when using the AskUI Hub as the model provider. | | `ASKUI_TOKEN` | Your workspace access token. Required when using the AskUI Hub as the model provider. See [Workspaces](/docs/account-billing/workspaces). | ## AgentOS connection [#agentos-connection] | Variable | Default | Description | | ------------------------------------------ | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------- | | `ASKUI_CONTROLLER_CLIENT_SERVER_AUTOSTART` | `true` | Set to `false` to prevent the CLI from starting a local AgentOS instance automatically. Use this when connecting to a remote AgentOS. | | `ASKUI_CONTROLLER_CLIENT_SERVER_ADDRESS` | `localhost:23000` | Address of the AgentOS gRPC server to connect to. Override when using a remote AgentOS instance. | # Glossary (/docs/reference/glossary) | Term | Definition | | ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Access token** | A credential that authenticates the CLI or an external integration with the AskUI Hub. Every token belongs to exactly one workspace and has an expiry date. See [Workspaces](/docs/account-billing/workspaces). | | **AgentOS** | The runtime layer that gives the AI agent direct OS-level control: keyboard, mouse, screen capture, and system functions. Runs on Windows, macOS, and Linux. | | **Annotation** | A marked region around a UI element with metadata including its name, text content, and bounding box coordinates. | | **Bounding box** | A rectangle defined by coordinates that describes the location and size of a UI element on screen. | | **Companion Mode** | An AgentOS control mode where AgentOS runs on an external device (e.g. Raspberry Pi) and controls the target through hardware interfaces. No software installation on the target is required. | | **Credentials** | The combination of a **workspace ID** and an **access token** for the same workspace. | | **Element** | A graphical user interface component: a button, text field, menu item, image, etc. | | **Host Mode** | An AgentOS control mode where AgentOS runs directly on the target machine and controls it via OS-level APIs. | | **Inference** | The process of sending a screenshot to the AI model to identify and locate UI elements. | | **Project** | A folder structure created by AskUI Desktop containing tests, context files (ui.md, rules.md), procedures, and plans. The unit of work for authoring and running tests. | | **Procedure** | A reusable multi-step sequence (e.g. "log in") stored in `procedures/` and invoked from test steps by name. | | **Run report** | The output of a test run: a Markdown file with a status and screenshot per step. Statuses are `PASSED`, `FAILED`, `WARN`, `SKIPPED`, or `BROKEN`. | | **Workspace** | The account-level container in the AskUI Hub that owns access tokens, usage events, and billing. Every AskUI account belongs to one or more workspaces. | | **Workspace ID** | The unique identifier for a workspace. Used together with an access token to authenticate with the AskUI Hub. | # Network requirements (/docs/reference/network-requirements) Allowlist per **machine role**: the machine that starts runs needs AskUI and the model endpoint, a test machine needs neither. How to use that split as a security boundary: [What the agent may do](/docs/best-practices/security#allowlist-the-domains-nothing-more). ## Outbound, from the machine that starts runs [#outbound-from-the-machine-that-starts-runs] AskUI Desktop and the [CLI](/docs/running-tests/cli) need HTTPS (443) to: | Domain | Purpose | Still needed if… | | ---------------------- | -------------------------------------------- | ------------------------------------------------------------------- | | `auth.askui.com` | Sign-in (OIDC) | Not in [license-key mode](/docs/get-started/licensing) | | `workspaces.askui.com` | Workspace API, access-token validation | Not in license-key mode | | `inference.askui.com` | Model inference through the AskUI hub | Not with [your own model provider](/docs/extending/model-providers) | | `files.askui.com` | Installer and update downloads (release CDN) | Only for installing and updating | With your own provider, that endpoint replaces `inference.askui.com`, your Anthropic account, your Azure AI Foundry resource, or a host inside your own network. With **on-premise inference** no external model domain is needed at all. Desktop's sign-in callback uses the hostname `desktop-auth.askui.com`, which resolves to `127.0.0.1`. No outbound connection is made to it, but the name must be **resolvable**: a DNS setup that blocks unknown external names breaks the callback even though nothing leaves the machine. ## Outbound, from a test machine [#outbound-from-a-test-machine] A machine that only hosts the application under test needs **nothing from this list**. It talks to the application under test; the machine driving the run does the rest. Downloads are the exception, and only until the machine is provisioned: browser bundles for [web profiles](/docs/devices/web-browser), the Android SDK for [emulators](/docs/devices/android-emulator), and NuGet packages for [custom tools](/docs/extending/custom-tools), all resolvable from an internal mirror. ## Inbound [#inbound] | Port | Where | When | | ----- | ------------------------------------------------------------------------------------- | ------------------------------------- | | 23000 | The machine running [AgentOS](/docs/agentos), started by the app | Driving that machine from another one | | 26000 | Same, when AgentOS runs as an [installed service](/docs/agentos/installation/service) | Shared lab machines | Restrict those ports to the machines allowed to start runs. Local runs on a single PC need no inbound rule at all, the connection never leaves the machine. ## Proxy [#proxy] AskUI Desktop is configured **in the app**, under **Settings → Network proxy** (system proxy, manual address with bypass list, NTLM/Kerberos or stored credentials). Setup and what to check when connections fail: [Network proxy](/docs/troubleshooting/network-proxy). ## Air-gapped [#air-gapped] [License-key mode](/docs/get-started/licensing) plus a model endpoint inside your network removes every AskUI domain from the picture. Runtime downloads still need a reachable mirror or pre-provisioned tooling. What that setup looks like end to end: [Enterprise fit](/docs/concepts/enterprise-fit). # Dashboard (/docs/results/dashboard) Where a [run report](/docs/results/run-report) covers a single execution, the **Dashboard** rolls every run in the project into one view, the closest thing to a test execution status report across your whole suite. Open the **Dashboard** page when you want the trend rather than one result. The Dashboard in AskUI Desktop showing total runs, average pass rate, per-status counters, and a test-status-over-time chart

The Dashboard in AskUI Desktop showing total runs, average pass rate, per-status counters, and a test-status-over-time chart

Section by section: * **In flight**: what is running or queued right now — green dot running, amber dot queued with the reason it waits. Links to the [Runs page](/docs/running-tests#watch-it-live). Shown only while something is active. * **Upcoming schedules**: the next few [schedules](/docs/running-tests/scheduling) by their due time. Shown only when schedules exist. * **The verdict**: one sentence on the latest run — did it pass, and when. * **Needs attention**: the failing or broken tests of the latest run, each a link into its report. * **Trend**: pass-rate over the recent runs, once there are at least two. * **Recent runs**: the run history, newest first. The numbers derive from each run's [summary report](/docs/results/run-report#summary-report), the counting rules live there. # Runs (/docs/results) Every run is kept: its reports, screenshots, and the complete conversation. The **Runs** page is where you open them. ## Open a run's results [#open-a-runs-results] 1. Open the **Runs** page. 2. Pick the run in the sidebar, newest first, one entry per run with its pass tally. 3. Choose what to look at: * **Conversation log**: the full execution record (below). * **Summary report**: the run-level table, one row per test. * **The per-test entries**: each test's [test report](/docs/results/run-report) with its status badge. **Open in Explorer** on a test jumps to its files on disk, the run's folder is one level up. ## Conversation log [#conversation-log] Beside the written reports, every run keeps its full conversation, each agent message, tool call, and screenshot in order. Where the report gives the result, the log is the complete execution record: open it when a status alone doesn't explain what happened. The conversation log of a finished run, agent messages, tool calls, and screenshots in order

The conversation log of a finished run, agent messages, tool calls, and screenshots in order

Reading it element by element: * **Numbered circle + TEST header**: one phase (setup, a test case, teardown) with its file path, status badge (PASS/FAIL), and how many tool calls it took, this is where you see each test. Expanding a phase reveals a collapsed **System prompt** section, the exact standing instructions the agent received for that phase. * **Text lines**: the agent's narration, what it sees on screen and what it decides to do next. * **Chips**: tool calls, a screenshot, a keystroke, a click, or a named tool like *Save screenshot* / *Write file*, with the result behind it. * **`350 → 26`**: tokens in → out for that step, the cost signal of the run. * **`500ms`**: the step's latency. During a run the same view streams live, [Watch it live](/docs/running-tests#watch-it-live). ## Share results [#share-results] To hand a result to a developer or attach it to a ticket, zip the run's timestamped folder (**Open in Explorer** on a test, one level up), the reports are plain Markdown with the step screenshots next to them, readable anywhere. ## On disk [#on-disk] Each run is one timestamped folder under `agent_workspace/`: ``` /agent_workspace/ └── 2026-07-23_21-56-55/ ← one folder per run ├── conversation.json ← the conversation (the app re-renders it) ├── summary_report.md ← run-level table └── open_notepad_windows/ ← one folder per test ├── open_notepad_windows_report.md ← the test report ├── step_1.png └── step_2.png ``` Run folders are independent of each other, archive or delete old ones as your evidence-retention policy requires; the Runs list and [Dashboard](/docs/results/dashboard) reflect whatever folders exist. The folder is [gitignored](/docs/projects#git-versioning--sync) by design. From the CLI, `--workspace` overrides the location. # Run report (/docs/results/run-report) Two written reports per run, the test report per case and the run-level summary. Where they live and how to open them: [Runs](/docs/results). ## Test report [#test-report] Each test gets its own `_report.md` (with a `step_.png` screenshot for every step), the agent writes it while it executes. It is the primary record of what the agent did, what it expected, and what it actually observed.

Test Case Report: login_test

Test Case ID: login_test
Date: 2025-11-12
Status: WARN

Summary

The agent successfully logged in using the QA credentials and verified the dashboard was shown.

Preconditions

1\. Application is running and the login page is visible.

Test Steps

1. Navigate to the login page , Status: PASSED

Agent Interpretation: Opened browser and navigated to the login URL.
Expected: Login form visible.
Actual: Login form visible with username and password fields.

step_1.png

2. Enter QA credentials and submit , Status: WARN

Agent Interpretation: Filled the form using the QA credential secrets.
Expected: Dashboard page shown after successful sign-in.
Actual: Dashboard loaded with the welcome banner, but only after roughly 15 seconds.

step_2.png

Postconditions

1\. User is signed in and the dashboard is visible.

Issues

Sign-in took about 15 seconds, well above the usual 2–3 seconds. The dashboard did load completely, so step 2 is WARN rather than FAILED, but this is worth watching before it becomes a timeout.

Conclusion

The login flow works, with one caveat: sign-in is unusually slow. Functionally PASSED behaviour, reported as WARN because of the response time.

```markdown # Test Case Report: login_test **Test Case ID:** login_test **Date:** 2025-11-12 **Status:** WARN ## Summary The agent logged in using the QA credentials and verified the dashboard was shown — but sign-in was unusually slow (about 15 seconds). ## Preconditions 1. Application is running and the login page is visible. ## Test Steps 1. **Navigate to the login page** — Status: **PASSED** - **Agent Interpretation:** Opened browser and navigated to the login URL. - **Expected:** Login form visible. - **Actual:** Login form visible with username and password fields. - ![Step 1](step_1.png) 2. **Enter QA credentials and submit** — Status: **WARN** - **Agent Interpretation:** Filled the form using the QA credential secrets. - **Expected:** Dashboard page shown after successful sign-in. - **Actual:** Dashboard loaded with the welcome banner — but only after roughly 15 seconds. - ![Step 2](step_2.png) ## Postconditions 1. User is signed in and the dashboard is visible. ## Issues Sign-in took about 15 seconds — well above the usual 2–3 seconds. The dashboard did load completely, so step 2 is WARN rather than FAILED, but this is worth watching before it becomes a timeout. ## Conclusion The login flow works, with one caveat: sign-in is unusually slow. Functionally PASSED behaviour, reported as WARN because of the response time. ``` Section by section: * **Test metadata**: who, when, and how it went. * **Test Case ID**: ties the report to the [test case](/docs/writing-tests), its title ID or file name. * **Date**: when the test executed. * **Status**: the overall result, the `**Status:**` line is what the [runner parses](/docs/writing-tests/agents-and-prompts#report-format) as the run's result. One of: * `PASSED`, actual result matches expected. * `FAILED`, the step executed but the result didn't match. * `SKIPPED`, intentionally not executed (e.g. precondition not met). * `WARN`, passed, but with a caveat (slow response, minor visual deviation). * `BROKEN`, could not execute: crash or infrastructure failure. The overall status is the worst of the steps: `BROKEN` if any step broke, else `FAILED` if any failed, else `WARN`, else `SKIPPED` if all skipped, otherwise `PASSED`. * **Summary**: the outcome in two sentences, read this first. * **Preconditions**: whether the assumed starting state was actually met, a test that began from the wrong state fails here, not misleadingly in a step. * **Test Steps**: one entry per step. * **Agent Interpretation**: how the agent understood your instruction, check this when a step surprises you. * **Expected** / **Actual**: the step's contract and what was actually observed. * **Status**: the step's own status, same vocabulary as above. * **Screenshot**: the step's evidence (`step_.png`). * **Postconditions**: whether the overall pass criterion held at the end. * **Issues**: anything worth a look that isn't a failure, the incident notes. In the example above: the 15-second sign-in. * **Conclusion**: the agent's closing assessment of the whole case. The report's shape is defined by your project's [report format](/docs/writing-tests/agents-and-prompts#report-format). What to do with a failing report, triage, root cause, where the fix belongs: [Analyzing failures](/docs/best-practices/analyzing-failures). *** ## Summary report [#summary-report] `summary_report.md` in the run folder summarizes the whole run: a totals head (test-case count, per-status counts, run duration), one Markdown table across all tests with status and execution time per test, and detail sections that quote each warned, failed, or broken test's issues from its own report. ```markdown # Summary Report **Date:** 2026-08-06 **Total:** 3 Test Cases - **✅ Passed:** 2 - **❌ Failed:** 1 - **⏱️ Duration:** 00:04:03 | Test Case ID | Test Case Name | Status | Duration | | --- | --- | --- | --- | | login_test | login_test | ✅ Passed | 00:01:12 | | settings_test | settings_test | ✅ Passed | 00:00:48 | | checkout_test | checkout_test | ❌ Failed | 00:02:03 | # ❌ Fails ## checkout_test The pay button never appeared after entering the voucher code. ``` The detail sections carry each test's `## Issues` (else `## Conclusion`) section, so a red run can be triaged from this one document; a test that ran but produced no report reads "No report written". Tests that a [test plan](/docs/writing-tests/plans) selected but the run never reached are listed as `💥 Broken` with a duration of `—`, so nothing goes missing silently. The Dashboard and Runs pages in AskUI Desktop derive their pass/fail counters from this table. `WARN` counts as passed (tracked separately in the warning total). `SKIPPED` tests never executed and are excluded from the total. `BROKEN` tests without an execution time in the summary (tests that never started) are also excluded from the total, they show in the table but don't affect the pass rate. In CI, upload the whole workspace folder as a build artifact so the reports and screenshots stay attached to the run even when tests fail: ```yaml title=".github/workflows/ui-tests.yml" - uses: actions/upload-artifact@v4 if: always() with: name: askui-run path: agent_workspace/ ``` # CLI reference (Coming Soon) (/docs/running-tests/cli) The CLI runs the exact Projects you build in AskUI Desktop. The same `tests/`, `plans/`, `procedures/`, and `utils/` drive both, author in the app, run anywhere with the CLI. See [Install the CLI](/docs/running-tests/install-cli) for setup and authentication. ```bash askui run [] [options] ``` Run from your Project root (or pass `--project-root`). Without arguments, `askui run` executes the whole tests folder against your local desktop (`tests/`, or the folder `askui.json` names as the tests path). ## What a run does [#what-a-run-does] For every test, the agent works in **phases**: the folder's `setup.md` (if present) runs first, then the tests, then `teardown.md`, with any `rules.md` applied throughout. Each phase is one autonomous agent execution that reads the test file, drives the UI, and writes a report with screenshots. When you target a nested folder tree, setups run top-down, teardowns bottom-up, and `rules.*` and `ui.*` accumulate per level, always starting at the tests root, so `tests/rules.md` and `tests/ui.md` apply to every run shape; a failing setup blocks the tests it guards. ### The target [#the-target] | Target | Behavior | | ------------- | ------------------------------------------------------------------------------------------------------- | | *(none)* | Runs the Project's `tests/` folder. | | A folder | Runs every test in the folder recursively, with per-folder setup/teardown lifecycle. | | A single file | Runs one test, its ancestor folders' setups still run first (top-down) and teardowns after (bottom-up). | Relative targets resolve against `--project-root`. Supported test file types: `.txt`, `.md`, `.pdf`, `.csv`, `.json`. ### Test plans [#test-plans] ```bash askui run --plan smoke ``` `--plan ` loads `plans/.yaml`, a [test plan](/docs/writing-tests/plans), and runs exactly the tests it selects, in plan order. The plan's device list is ignored — the CLI runs on the surface its flags configure and prints a note saying so. A plan that selects no tests, or does not exist, is a configuration error (exit code 2). ## Run options [#run-options] | Option | Default | Description | | ---------------------- | ------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------- | | `--plan ` | , | [Test plan](/docs/writing-tests/plans) (without extension) from the Project's `plans/` directory; its device list is ignored. | | `--project-root ` | current directory | The Project root (`tests/`, `procedures/`, `plans/`, `utils/`). | | `--workspace ` | `/agent_workspace/` | Output directory for reports, screenshots and the conversation. | ## Surface options [#surface-options] When the project's `devices.json` has exactly one profile matching the surface options, its name and [prompt text](/docs/writing-tests/agents-and-prompts#device-prompt) join the agent's system prompt, the same device knowledge as an in-app run. | Option | Default | Description | | -------------------------- | ------------------ | ----------------------------------------------------------------------------------------------------------------------------------- | | `--surface ` | `agent-os` | What the agent drives: `agent-os` (desktop), `web` (browser), or `android`. | | `--controller-host ` | `localhost` | [AgentOS](/docs/agentos) host (`agent-os` only). A local (loopback) AgentOS is started automatically; remote hosts are attached to. | | `--controller-port ` | `23000` | AgentOS gRPC port (`agent-os` only). | | `--display ` | `1` | Display to operate on (`agent-os` only). | | `--device-serial ` | first `adb` device | Android device serial (`android` only). | ```bash # Drive a remote machine's AgentOS on its second display askui run --controller-host 192.168.1.42 --display 2 # Run the suite against a browser askui run --surface web # Run against a specific Android device askui run --surface android --device-serial emulator-5554 ``` ## Model options [#model-options] By default, inference runs through the **AskUI hub**, billed to your workspace and authenticated with `ASKUI_WORKSPACE_ID`/`ASKUI_TOKEN`, nothing to configure. The hub serves more than one model. | Option | Default | Description | | ---------------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | | `--model-provider
` | `askui` | `askui`, `anthropic`, `openai`, or `openai-compatible`. | | `--model-id ` | provider default | The model to use. With `askui`, selects the hosted model, e.g. `gemini-3.5-flash`; default is the workspace's Claude model. | | `--api-key ` | , | API key (BYOM providers only). | | `--endpoint-url ` | , | Custom base/endpoint URL. Required for `openai-compatible`; a URL ending in `/chat/completions` is used as-is, a base URL gets the path appended. | ```bash # Hosted models on the AskUI hub askui run # workspace default (Claude) askui run --model-id gemini-3.5-flash # hosted Gemini # Bring your own model askui run --model-provider anthropic --api-key $ANTHROPIC_API_KEY askui run --model-provider openai-compatible \ --endpoint-url http://localhost:8000/v1 \ --model-id Qwen/Qwen3-VL-30B-A3B-Instruct ``` The agent drives the UI from **screenshots**, so BYOM models must be **vision-capable**: a text-only model will connect but can't see the screen. ## Credentials and `.env` [#credentials-and-env] The CLI loads a `.env` file from the Project root at the start of every run, variables already set in the environment win. This carries the hub credentials (`ASKUI_WORKSPACE_ID`, `ASKUI_TOKEN`) and anything your own [helper tools](#helper-tools) read. Treat access tokens like passwords: in CI, inject them from the secret store instead of committing a `.env` file. See [Workspaces & access tokens](/docs/account-billing/workspaces). ## Helper tools [#helper-tools] The CLI extends the agent with your Project's custom tools: every `*.dll` in `/helpers/` is loaded, and each public tool class in it becomes available to the agent next to the built-in ones. A broken assembly never takes down the run, the problem is reported and the rest keeps loading. ## What a run produces [#what-a-run-produces] Each run gets its own workspace folder (see `--workspace`), containing: * **Per-test artifacts**: `/_report.md` with the result and step-by-step details, plus `step_.png` screenshots. * **`summary_report.md`**: the run at a glance — totals with per-status counts and run duration, one table over all tests with status and duration each, and detail sections quoting every warned, failed, or broken test's issues. Tests a plan selected but the run never reached appear as broken, so nothing goes missing silently. * **`conversation.json`**: the full agent conversation, which AskUI Desktop re-renders as the [conversation log](/docs/results#conversation-log). The console shows each phase as it starts and finishes (`[test] tests/login.md -> PASSED`) and ends with the totals. ## Exit codes [#exit-codes] | Code | Meaning | | ----- | -------------------------------------------------------------------------------------- | | `0` | All executed tests passed. | | `1` | At least one test failed or broke, or the run produced no result. | | `2` | Usage or configuration error (bad flags, missing project/target, unreachable surface). | | `130` | Interrupted (Ctrl+C), partial reports and the summary are preserved. | ## Running in CI [#running-in-ci] The exit codes make `askui run` CI-native: the job fails exactly when tests fail. A minimal GitHub Actions job: ```yaml title=".github/workflows/ui-tests.yml" jobs: ui-tests: runs-on: [self-hosted, askui] # a runner with AgentOS installed env: ASKUI_WORKSPACE_ID: ${{ vars.ASKUI_WORKSPACE_ID }} ASKUI_TOKEN: ${{ secrets.ASKUI_TOKEN }} steps: - uses: actions/checkout@v4 - run: askui run --plan smoke --workspace ./run-artifacts - uses: actions/upload-artifact@v4 if: always() with: name: askui-run path: ./run-artifacts ``` Pin `--workspace` to a fixed path so the artifact upload finds the reports, and upload with `if: always()` so failing runs keep their evidence. For setting up the machine under test, see [AgentOS CI deployment](/docs/agentos/deployment/ci). # Run a test (/docs/running-tests) ## Start a run [#start-a-run] 1. Open the test on the **Tests** page. 2. Pick a connected device in the **Run on** dropdown (see [Devices](/docs/devices)). 3. Press **Run**, or use its split menu to run the whole folder. For a saved multi-test selection, run a [test plan](/docs/writing-tests/plans) from the **Test Plans** page instead. ## If it can't start right away [#if-it-cant-start-right-away] A run needs its device to itself, and the app runs at most **two runs in parallel** (on different devices — two runs never share a device). When you press Run and the run cannot start immediately, a dialog asks what to do: * **Queue run**: the run waits its turn and starts automatically as soon as the device is free and a parallel slot opens. A queued run gives up after 30 minutes of waiting, with the reason recorded in its result. * **Don't start**: nothing is queued, nothing runs. Queued runs appear in the Runs sidebar under **Queued**, each with the reason it waits; the ✕ on the row cancels it. Runs started by a [schedule](/docs/running-tests/scheduling) skip the dialog and queue silently. Closing the app discards the queue — you are warned first when anything is still queued or running. ## What a run does [#what-a-run-does] The agent works in phases: the folder's `setup.md` first, then the tests, then `teardown.md`, with `rules.md` applied throughout. Each phase is one agent execution: read the file, drive the UI, write a report with screenshots. A failing setup records its tests as broken, the entry criteria weren't met. ## Watch it live [#watch-it-live] The conversation is the live test execution log, every message, tool call, and screenshot as it happens. Use it to catch unexpected behaviour before the report exists. 1. Open the **Runs** page. 2. Pick the run in the runs sidebar on the left, it lists every run of the project, and **several runs can be active at once**; each streams its own conversation. 3. Follow along, the controls at the top of the conversation: * **Autofollow**: keeps the view pinned to the newest entry while the run streams; scroll up to inspect something and it pauses, click it to snap back. * **Quiet**: just the story: the agent's narration, minimal noise. * **Detailed**: the default: narration plus every tool call with its result, tokens, and latency. * **Raw**: the underlying messages as JSON, for debugging. ### Reading the conversation [#reading-the-conversation] * **Numbered circle + TEST header**: one phase (setup, a test case, teardown) with its file path, status badge (PASS/FAIL), and how many tool calls it took, this is where you see each test. Expanding a phase reveals a collapsed **System prompt** section, the exact standing instructions the agent received for that phase. * **Text lines**: the agent's narration, what it sees on screen and what it decides to do next. * **Chips**: tool calls, a screenshot, a keystroke, a click, or a named tool like *Save screenshot* / *Write file*, with the result behind it. * **`350 → 26`**: tokens in → out for that step, the cost signal of the run. * **`500ms`**: the step's latency. The same view stays available after the run finishes, see the [conversation log](/docs/results#conversation-log) on the results page. After the run: [read the report](/docs/results/run-report). Recurring runs: [Scheduling](/docs/running-tests/scheduling). Headless / CI: [CLI](/docs/running-tests/cli). # Install the CLI (Coming Soon) (/docs/running-tests/install-cli) The AskUI CLI (`askui`) runs the same Projects you author in AskUI Desktop, without the UI. One command, `askui run`, executes your tests against a desktop, a browser, or an Android device and writes the same reports the app produces. Use it locally for scripted runs, in CI, or on a schedule. **Install the CLI** *(coming soon)* The CLI is **not yet publicly available**, the installer is in progress. This page documents what running it looks like so you can plan your CI setup; the steps below apply once the CLI ships. **Verify the install** ```bash askui --help ``` You should see the `askui run` usage with its options. **Set up the surface you want to drive** Desktop runs need [AgentOS](/docs/agentos) on the machine under test. On your own machine, the CLI **starts a local AgentOS automatically** for the run; if AgentOS is installed as a [service](/docs/agentos/installation/service), the CLI detects and attaches to it instead. For a remote machine, install AgentOS there and point the CLI at it with `--controller-host`. Nothing to install up front: on the first `--surface web` run, the CLI downloads the browser it drives (Chromium via Playwright) automatically. `adb` must be on your `PATH` and the device (or emulator) visible to `adb devices`. See the [ADB setup guide](/docs/agentos/deployment/local#android-setting-up-adb). **Authenticate** With the default model provider (the AskUI hub), the CLI reads two environment variables: ```bash export ASKUI_WORKSPACE_ID="" export ASKUI_TOKEN="" ``` Create the access token in the [AskUI Hub](https://hub.askui.com), see [Workspaces & access tokens](/docs/account-billing/workspaces). Instead of exporting, you can also put both lines in a `.env` file in your Project root; the CLI loads it on every run (variables already set in the environment win). **Run a Project** ```bash cd my-project askui run ``` See [Running from the CLI](/docs/running-tests/cli) for the full command reference, test plans, surfaces, model selection, and CI usage. # Scheduling (/docs/running-tests/scheduling) A schedule runs a test or [test plan](/docs/writing-tests/plans) without you pressing Run. A test runs on the device profile you pick in the dialog; a plan runs on the device recorded in the plan. A scheduled run whose device is busy [queues silently](/docs/running-tests#if-it-cant-start-right-away) and starts when the device is free. Schedules fire only while AskUI Desktop is running, it is not a background service. Close the app, or let the machine sleep or shut down, and the scheduled run silently does not happen: no firing, no report, and no catch-up when the app comes back. For runs that must never be missed, the nightly regression, use the [CLI](/docs/running-tests/cli) on a build server with your CI scheduler. ## Manage all schedules in one place [#manage-all-schedules-in-one-place] The **Runs** page's sidebar has two views, **Runs** and **Schedules**. The Schedules view lists every schedule of the project permanently, each with an on/off toggle, a pencil to edit its cadence, and delete. The **+** button creates a new one: pick any plan or test of the project in **What to run**. The Dashboard shows the next few upcoming schedules at a glance. ## Create a schedule [#create-a-schedule] 1. Open the test on the **Tests** page (or the plan on **Test Plans**). 2. Click the alarm-clock button, the **Scheduled runs** dialog opens, prefilled with this file. 3. Pick the **Device profile** (for a plan this step disappears, the plan's own devices are used). 4. Pick the **Cadence**: **Daily at a time** (with weekdays) or **Every N minutes**. 5. Click **Add schedule**. The dialog lists every schedule in the project with its last outcome, the run's results, or why it was skipped (e.g. the device could not connect). Schedules live in the project's `schedules.json`. A due schedule never double-fires, even when a run outlasts its interval. ## Pause or delete a schedule [#pause-or-delete-a-schedule] The dialog lists every schedule in the project: 1. Untick the **on** checkbox to pause a schedule, it stays configured but stops firing (the row dims). 2. Click the trash button (**Delete schedule**) to remove it. There is no edit: to change the cadence or device, delete the schedule and add it again. # Get help (/docs/support/get-help) ## Before reaching out [#before-reaching-out] Working through these steps first usually resolves the issue faster than opening a ticket: **Check the run report.** Open the failing run's report and look at the step that failed. The screenshot and status (`FAILED` vs `BROKEN`) usually point directly at the problem. See [Reading a run report](/docs/results/run-report). **Check the troubleshooting guide.** The most common AgentOS and connectivity issues are documented in [Troubleshooting](/docs/troubleshooting). ## Diagnostic report [#diagnostic-report] We are building a diagnostic tool that automatically checks your system configuration, installed versions, network connectivity, AgentOS reachability, environment variables, and more, and writes a structured report. You will be able to send this report directly to support so we can tell you exactly what is wrong, for example why a remote AgentOS cannot be reached. ## What to include in a bug report [#what-to-include-in-a-bug-report] A good bug report gets you a faster answer. Include: * **AskUI Desktop version**: shown in the app's About screen * **Operating system and version**: e.g. Windows 11 22H2, macOS 14.4 * **Steps to reproduce**: the exact sequence of actions that causes the issue * **Expected behavior**: what should happen * **Actual behavior**: what actually happens, including any error messages * **The run report**: the `.md` file or a screenshot of the failing step * **AgentOS logs** (for AgentOS issues), see [Logs](/docs/troubleshooting#logs) ## Contact channels [#contact-channels] | Channel | Best for | Response time | | ----------------------------------------------- | ------------------------------- | -------------------- | | [support@askui.com](mailto:support@askui.com) | Bug reports, technical issues | 24-48 business hours | | [feedback@askui.com](mailto:feedback@askui.com) | Feature requests, docs feedback | - | Mark your email subject **URGENT** for critical production issues. ## Enterprise support [#enterprise-support] Enterprise customers have access to: * A dedicated support channel * Priority response times * Direct access to the engineering team * Custom onboarding and training sessions Contact your account manager to activate enterprise support. ## Status and known issues [#status-and-known-issues] * **Service status:** [status.askui.com](https://status.askui.com) * **Security and compliance:** [trust.askui.com](https://trust.askui.com) * **GitHub issues:** [github.com/askui](https://github.com/askui) ## Community resources [#community-resources] * [YouTube](https://youtube.com/@askui), tutorials and walkthroughs * [Blog](https://blog.askui.com), product updates and deep-dives # Support (/docs/support) Answers to common questions about platform support, pricing, security, and on-premise options. How to report bugs, what to include, and where to reach the team and community. Step-by-step fixes for common AgentOS and connectivity issues. ## Status and trust [#status-and-trust] | Resource | Link | | -------------------------------------- | -------------------------------------------- | | Service status | [status.askui.com](https://status.askui.com) | | Security and compliance (Trust Center) | [trust.askui.com](https://trust.askui.com) | ## Contact [#contact] | Channel | Use for | | ----------------------------------------------- | ------------------------------------------- | | [support@askui.com](mailto:support@askui.com) | Bug reports and technical issues | | [feedback@askui.com](mailto:feedback@askui.com) | Feature requests and documentation feedback | # AgentOS service issues (Windows) (/docs/troubleshooting/agentos-service) The steps on this page apply to AgentOS running as a **Windows system service** . For standalone mode on macOS or Linux, check the terminal output for errors. ### Restart the AgentOS service [#restart-the-agentos-service] Many issues are resolved by restarting the service. To restart it via Task Manager: 1. Open **Task Manager** (`Ctrl+Shift+Esc`). 2. Switch to the **Services** tab (the puzzle-piece icon). 3. Search for **`AskuiCoreService`**. 4. Right-click the entry and choose **Restart**. ### Agent can't press CTRL+ALT+DEL [#agent-cant-press-ctrlaltdel] The login/lock screen cannot be automated unless the local security policy allows services to simulate a Secure Attention Sequence (SAS). To enable it: 1. Open the **Group Policy Editor** (`gpedit.msc`). 2. Navigate to **Computer Configuration → Administrative Templates → Windows Components → Windows Logon Options**. 3. Set **Disable or enable software Secure Attention Sequence** to **Enabled**. Also verify that `SERVICE_ENABLE_SAS` is set to `1` (default) in the [Silent installer parameters](/docs/agentos/installation/silent/#parameters). The service includes a `windows.forceSas` option that overrides group policies. This is **not recommended** as it may trigger system integrity checks. Configure the GPE policy instead. ### Agent can't interact with apps running as Administrator [#agent-cant-interact-with-apps-running-as-administrator] Windows prevents non-elevated processes from sending input to elevated (Run as Administrator) applications. If your agent can't click or type into an app running as Administrator, you need to start the Remote Device Controller in elevated mode. To enable elevated mode: * **Service Installer (GUI)**: Open **Windows Add/Remove Programs**, find AskUI AgentOS, click **Modify**, and enable the elevated execution engine option. Alternatively, reinstall AgentOS with the option enabled. * **Silent Installer**: Set `SERVICE_EXECUTION_ENGINE_ELEVATED` to `1`, see [Silent installer parameters](/docs/agentos/installation/silent/#parameters). Running the execution engine elevated grants it higher privileges. Only enable this when you need to automate applications that run as Administrator. ### `ipv4:127.0.0.1:23000 unavailable` in Standalone mode [#ipv412700123000-unavailable-in-standalone-mode] If the client fails to connect with an error like `ipv4:127.0.0.1:23000 unavailable` while running in **Standalone** runtime mode, autostart of the local AgentOS instance is likely disabled. Check the `ASKUI_CONTROLLER_CLIENT_SERVER_AUTOSTART` environment variable, if it is set to `false`, the client will not start AgentOS automatically and there is nothing listening on port `23000`. Either unset the variable or set it to `true`: ```powershell $env:ASKUI_CONTROLLER_CLIENT_SERVER_AUTOSTART="true" ``` ```bash export ASKUI_CONTROLLER_CLIENT_SERVER_AUTOSTART=true ``` This variable is intentionally set to `false` when connecting to a remote AgentOS instance, see [Remote Windows VM](/docs/agentos/deployment/ci/#remote-windows-vm). ### Need to reinstall or remove [#need-to-reinstall-or-remove] **Do not** stop or remove the service from Windows Services manually. Always **uninstall the full AskUI AgentOS application** via Windows Add/Remove Programs. This ensures proper cleanup so the client falls back to standalone mode. # Connection & sessions (/docs/troubleshooting/connection-sessions) ### Connection error or session mismatch at run start [#connection-error-or-session-mismatch-at-run-start] Connection errors and session mismatches happen when AgentOS and Desktop have lost sync, most often after a machine restart or sleep cycle. **Correct restart order for multi-machine setups:** 1. Restart AgentOS on the target machine(s) first. 2. Then reconnect from AskUI Desktop. If the error persists, use the **Restart** button on the Devices page in the Desktop sidebar to force a clean session reset. If the problem recurs after restarts, check whether the target machine received a new IP address from DHCP, see the entry below. ### Tests won't start after a reboot or power outage [#tests-wont-start-after-a-reboot-or-power-outage] Work through this checklist: 1. **Verify all services are running.** Open the Devices page in AskUI Desktop and confirm every connected machine shows a green status. 2. **Check for IP address changes.** DHCP can reassign addresses after a restart, breaking a stored connection. Use hostnames instead of static IPs wherever possible. 3. **Look for stale configuration.** Old connection settings in your project or workspace config may point to a previous address. Update or remove them and re-add the device. ### Run stops when RDP session is disconnected or screen is locked [#run-stops-when-rdp-session-is-disconnected-or-screen-is-locked] * **Remote Desktop (Windows), session disconnect:** Disconnecting an RDP session suspends the virtual display. AgentOS in Host Mode loses its view of the screen and the run ends. Either keep the RDP session connected and the window visible during the run, or run AgentOS as a **system service** so it operates independently of the interactive session, see [Running as a service](/docs/agentos/installation/service). * **Remote Desktop (Windows), window minimized:** Minimizing the RDP client window (`mstsc`) simulates a display disconnect on the remote machine. The Remote Device Controller loses access to the display and may return empty screenshots or shut down. To prevent this, set a registry value on the machine running the RDP client: Open `regedit`, navigate to `HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Terminal Server Client`, and create a `DWORD` value named `RemoteDesktop_SuppressWhenMinimized` with data `2`. * **Locked screen:** A locked session hides the display on both Windows and macOS. Either keep the machine unlocked during unattended runs, or use a virtual display tool that persists across lock events. For fully unattended CI scenarios, the recommended setup is AgentOS as an OS service, see [CI/CD integration](/docs/agentos/deployment/ci). # Troubleshooting (/docs/troubleshooting) Session mismatches, post-reboot failures, RDP and locked-screen issues. Wrong clicks, screen misinterpretation, and runs stopping mid-test. Unexpected cost increases and slow or crashing long runs. Restarting the service, CTRL+ALT+DEL, elevated apps, and port errors. Configuring the proxy AskUI Desktop uses, and what to check when connections fail. Debugging FAILED and BROKEN statuses, and runs that produce no results. 429, 401, 403, 404, connection refused, and vision-model errors for BYOM. *** ## Logs [#logs] The AskUI Core Service and the Remote Device Controller write to two separate locations. For most issues, collect logs from **both**: **Remote Device Controller** (runs in the interactive user session): ``` %userprofile%\.askui\Logs ``` **AskUI Core Service** (runs as SYSTEM): ``` C:\Windows\System32\config\systemprofile\.askui\logs ``` # Misclicks & stuck runs (/docs/troubleshooting/misclicks-and-stuck-runs) ### Agent clicks the wrong element or misses small buttons [#agent-clicks-the-wrong-element-or-misses-small-buttons] Vague element descriptions are the most common cause. Make descriptions specific: * **Use spatial anchors:** "the button in the bottom-right corner, below the Save dialog" is far more reliable than "the button". * **Include distinguishing features:** label text, color, icon shape, or position relative to a unique landmark on the screen. * **For very small targets:** explicitly instruct the agent to zoom before acting, for example, "zoom in on the toolbar area, then click the small red X icon". See [Prompting best practices](/docs/best-practices/prompting-best-practices) for a full guide on writing reliable element descriptions. ### Agent misinterprets a screen state [#agent-misinterprets-a-screen-state] The agent has no built-in knowledge of your application's UI. Add that knowledge to your project's `tests/ui.md`. Describe screens the agent commonly confuses, loading overlays, login walls, modal dialogs, error banners. Example: ``` A grey semi-transparent overlay with a spinner in the centre means the page is loading. Wait for it to disappear completely before taking any action. ``` See [Agents and prompts](/docs/writing-tests/agents-and-prompts) for the full prompt structure. ### Agent stops in the middle of a long test [#agent-stops-in-the-middle-of-a-long-test] Each run has a step limit that prevents runaway executions. If a test hits that limit, it ends with status `BROKEN`. To avoid this: * **Extract procedures:** move repeated sequences (e.g. "log in") into `procedures/` and call them by name from your tests. * **Split large tests:** break a long end-to-end flow into smaller focused test cases, each with its own clear goal. See [Project structure](/docs/projects) for how procedures and test files are organised. # Model provider errors (/docs/troubleshooting/model-providers) These errors appear when AskUI Desktop tests your API credentials on save, or during a run when the model provider rejects a request. They only apply when you have configured a [custom model provider](/docs/extending/model-providers). ### 429 Too Many Requests [#429-too-many-requests] **What it means:** You've sent too many requests to the AI provider in a short time. Their system is temporarily blocking new requests. Each provider (Anthropic, OpenAI, etc.) limits how many requests you can make per minute. A test suite that runs many tests back-to-back can hit this limit quickly. **What to do:** 1. Wait a few minutes, then try again, the limit resets automatically. 2. Check your quota in your provider's console: * Anthropic: [console.anthropic.com](https://console.anthropic.com) → Usage * OpenAI: [platform.openai.com](https://platform.openai.com) → Usage 3. If this happens regularly, upgrade to a higher usage tier with your provider. 4. Alternatively, switch back to the **AskUI hub** (Settings → Account → Model Provider → Remove key), the hub handles rate limits automatically. ### 401 Unauthorized [#401-unauthorized] **What it means:** The API key you entered is wrong, was typed incorrectly, or has been revoked. **What to do:** 1. Go to **Settings → Account → Model Provider** and check the key you entered. 2. Generate a fresh key in your provider's console and paste it in, don't type it by hand. 3. Make sure you copied the full key. Keys are long strings and it's easy to miss characters at the start or end. ### 403 Forbidden [#403-forbidden] **What it means:** Your API key is valid, but your account doesn't have permission to use this model or feature. Some models (e.g. Claude, GPT-5.4) require a paid plan or explicit access request. **What to do:** 1. Log in to your provider's console and check whether your plan includes API access to the model you selected. 2. Try a different model that's included in your current plan, or switch back to the AskUI hub. ### 404 Not Found / Model not found [#404-not-found--model-not-found] **What it means:** The model ID you entered doesn't exist, or was typed incorrectly. Model IDs are case-sensitive and must match exactly. **What to do:** 1. Go to **Settings → Account → Model Provider** and check the **Model** field. 2. Copy the exact model ID from your provider's documentation: * Anthropic: [docs.anthropic.com/models](https://docs.anthropic.com/en/docs/about-claude/models/overview) * OpenAI: [platform.openai.com/docs/models](https://platform.openai.com/docs/models) ### Connection refused / Could not connect to server [#connection-refused--could-not-connect-to-server] **What it means:** AskUI Desktop tried to reach your model's endpoint but got no response. Either the address is wrong or the server isn't running. Most common with self-hosted models (vLLM, Ollama, llama.cpp). **What to do:** 1. Make sure your local model server is running before starting a run. 2. Check the **Base URL** in **Settings → Account → Model Provider**, it should look like `http://localhost:8000/v1`. Confirm the port matches what your server uses. 3. If connecting to a remote server, check that it's reachable on your network and no firewall is blocking the port. ### The agent can't see the screen (text-only model) [#the-agent-cant-see-the-screen-text-only-model] **What it means:** The model you configured is text-only. It can't process screenshots, so the agent can't see the UI it's supposed to test. **What to do:** Switch to a **vision-capable** model: * Anthropic: any Claude 3 or later model * OpenAI: `gpt-5.4`, `gpt-5.4-mini` * Self-hosted: Qwen-VL, LLaVA, InternVL, or similar vision models # Network proxy (/docs/troubleshooting/network-proxy) The proxy is configured **in the app**: 1. Open **Settings → Network proxy**. (During license setup, the same form is in the wizard under **Behind a proxy? Configure network access**.) 2. Choose the **Proxy** mode: * **Use system proxy (recommended)**: the Windows proxy/PAC configuration applies. * **Manual proxy address**: enter the fields: * **Proxy address**: e.g. `http://proxy.corp.example:8080` * **Bypass for**: hosts reached directly, separated by semicolons (e.g. `*.corp.example; 10.0.0.*`). Localhost is always bypassed. * **No proxy (direct connection)**. 3. Choose **Proxy authentication**: * **Automatic (recommended)**: if the proxy asks for sign-in (HTTP 407), your current OS account answers automatically (NTLM / Kerberos). No password is stored, and nothing is sent unless the proxy actually asks. * **Username and password**: stored encrypted on this machine; it never leaves it. * **Never authenticate**. ### Connections still fail [#connections-still-fail] 1. Check the internet connection first, an interrupted connection produces the same errors as a blocked one. 2. Confirm the domains AskUI needs are reachable through the proxy: [Network requirements](/docs/reference/network-requirements). 3. If they are allowlisted and it still fails, your network is doing something to the traffic that needs your IT team, [get in touch](/docs/support/get-help) and we will work it out with them. # Performance & cost (/docs/troubleshooting/performance-cost) ### Run cost increased unexpectedly [#run-cost-increased-unexpectedly] Common drivers: | Driver | What to check | | ------------------------ | --------------------------------------------------------------------------------------------------------------------------- | | Oversized images or logs | Very large screen captures or log files raise the token count per step. | | Retry loops | A failing step the agent retries many times multiplies cost. Look for high step counts on single actions in the run report. | Open the run report and check the token count per step, the costliest steps are usually the ones to optimize. For a cross-run view, see the [Usage dashboard](/docs/account-billing/usage-dashboard). ### Run got slower or crashed during a very long execution [#run-got-slower-or-crashed-during-a-very-long-execution] Log and trace files grow continuously over a run. Very large files can exhaust memory or significantly slow the host machine. Mitigations: * Keep test cases short and focused, split a large suite into smaller cases, each producing its own report. * Archive or delete old run folders under `agent_workspace/` in your project periodically. * If a run must be long, monitor disk and memory usage on the AgentOS host and set up log rotation if your OS supports it. # Run failures (/docs/troubleshooting/run-failures) ### Debugging a failing run [#debugging-a-failing-run] The fastest way to debug a failing run: 1. Open the run report (the `.md` file in your project's workspace folder, or inline in AskUI Desktop). 2. Find the first step with status `FAILED` or `BROKEN`, that is where to focus. 3. Check the screenshot for that step. `FAILED` means the expected outcome didn't match; `BROKEN` means an infrastructure problem (connection lost, session expired) stopped the run. 4. For AgentOS issues, check the [logs](/docs/troubleshooting#logs) for more detail. If you cannot resolve the issue yourself, see [Get help](/docs/support/get-help) for how to generate a diagnostic report and contact support. ### Run completes with 0 results ("No tests ran") [#run-completes-with-0-results-no-tests-ran] The run started but found no test files to execute. 1. Check that you're running from the correct **Project root**, the folder that contains `tests/`, `plans/`, etc. 2. Check that your test files are inside `tests/` and have a supported extension (`.md`, `.txt`, `.pdf`, `.csv`, `.json`). 3. If you used `--plan`, check that the plan file exists in `plans/` and references valid test paths. # The system prompt (/docs/writing-tests/agents-and-prompts) The system prompt is what you'd tell a new tester on day one, standing instructions the agent gets before every run. Each part lives where it belongs, versioned with your testware: | Part | Lives in | You edit it? | | --------------------------------- | ------------------------------------ | ----------------------------------------- | | Capabilities and error discipline | Built into the app | No, it keeps runs honest | | Device prompt | Each device profile (`devices.json`) | Only when the machine has quirks | | UI information | `tests/ui.md` | Yes, the file you invest the most in | | Report format | `utils/format.md` | When your reporting needs differ | | Rules | `tests/rules.md` (and per suite) | Yes, your rules override everything above | ## Capabilities [#capabilities] The agent's identity and its error discipline are built in and not editable. The defaults are deliberately strict: > You have a maximum of **2 attempts per step**. If a step does not succeed > after 2 attempts, stop immediately. Do not try a third time. Two conventions the agent relies on: **Credentials** never live in a test. A step references a [named secret](/docs/extending/secrets); the agent types a placeholder and the real value is substituted at execution time, hidden from the model. \ **Scratchpad**: the agent can persist notes across phases with the scratchpad tools, but only when a test explicitly instructs it to. When the built-in behaviour doesn't fit your suite, don't look for a file to edit, write a [rule](#rules): rules are appended last and take precedence over the built-in instructions on conflicts. ## Device prompt [#device-prompt] Machine quirks the agent can't see for itself live on the device profile, keep it small. The agent already sees the screen, so OS and display basics rarely need describing. 1. Open **Devices** and click **Edit** on the profile. 2. Fill the **Prompt** field: ```markdown * Always execute on display 1. * The corporate lock screen appears after 5 minutes of inactivity. ``` The text is stored with the profile in `devices.json` and goes into the system prompt of every run on that device. Retargeting a project at a different machine means picking a different profile, its prompt travels with it. ## UI information [#ui-information] `tests/ui.md`, where the agent learns your application, the domain knowledge that lets a [colleague without domain knowledge](/docs/writing-tests/step-constructs) execute your tests. The default ships as placeholders: ```markdown ## Navigation The application has these primary areas: Dashboard, Records, Settings… ## State indicators "Login state is shown top-right: a green dot means signed in, red means the session expired — sign in again." ## Dialogs & quirks "Destructive actions open a centered confirmation dialog that must be confirmed or dismissed before continuing." ``` This is also where you steer **how** the agent operates your UI, "enter numbers via the on-screen numpad, never the keyboard" ([more](/docs/writing-tests/step-constructs#beyond-steps)). A suite that tests a different part of the system can carry its own `ui.md`: like `rules.md`, the files accumulate top-down, the top-level `tests/ui.md` first, then each folder's on the way to the test. `ui.md` is never executed as a test, the Tests page marks it **Context only**. ## Report format [#report-format] `utils/format.md`, the template for every report the agent writes: title, **Test Case ID**, date, status, and the per-step sections. Edit it under **Extending → Report Format**, it applies to all runs of the project. Two parts are load-bearing: * The **`**Status:**` line is parsed by the runner** to set the run's result, remove it and a passing test is recorded as BROKEN. * The status definitions (PASSED / FAILED / SKIPPED / WARN / BROKEN) are the [status vocabulary](/docs/results/run-report#test-report) your reports and Dashboard use. Customize the sections when your organization needs different evidence, keep the `**Status:**` line. ## Rules [#rules] `tests/rules.md` holds your standing corrections, appended at the very end of the system prompt behind an explicit note that **your rules take precedence** where they conflict with anything above. Instructions that apply to one suite only go into that folder's [rules.md](/docs/writing-tests/setup-teardown-rules#rules); the files accumulate top-down at run time. ## Checking what the agent got [#checking-what-the-agent-got] Every run records the exact system prompt per phase: open the run's conversation log and expand the **System prompt** section under an act header. What you read there is what the agent read. ## Upgrading an older project [#upgrading-an-older-project] Projects created before this layout carry a `prompts/` folder. Opening one shows the **Migrate Project** dialog: 1. Click **Migrate**. `ui_information.md` becomes `tests/ui.md`, `report_format.md` becomes `utils/format.md`, and the folder is renamed to `prompts_legacy/`, nothing is deleted, existing files are never overwritten. 2. If you had customized `system_capabilities.md`, the dialog offers to append **your additions** (not the standard text) to `tests/rules.md`, where they keep working, and now take precedence. 3. Text you wrote in `device_information.md` is not migrated automatically: copy it into the device profile's **Prompt** field on the Devices page. The original stays in `prompts_legacy/`. **Not now** dismisses the dialog for this project; the legacy files are simply no longer read. # Test cases (/docs/writing-tests) Test cases live as files under `tests/` and are managed on the **Tests** page: on the left the file tree with all test cases, on the right the editor with the opened test case and its Run controls. ## The concepts [#the-concepts] Six building blocks make up everything you author: All of it lives in one [Project](/docs/projects), plain files, shared via Git. A **suite** frames its test cases with [setup, teardown, and rules](/docs/writing-tests/setup-teardown-rules); how suites nest and scope chains is on [Organisation & Scope](/docs/writing-tests/organisation). Cases and setups call [procedures](/docs/writing-tests/procedures) for repeated sequences, and reference [secrets](#use-secrets-in-a-test) instead of containing credentials. [Test plans](/docs/writing-tests/plans) select which cases run together, and on which device. ## Create a test case [#create-a-test-case] A test case is one file: preconditions, steps with expected results, postconditions. The file name is the test's name in runs and reports. 1. Right-click in the file tree → **New Markdown** (or **New CSV**). 2. Name the file and fill the seeded template. How to phrase steps well: [Writing good tests](/docs/best-practices/writing-good-tests). The same menu holds everything on this page: creating files and folders, the quick-create entries for setup / teardown / rules, **Run on**, and **Duplicate / Rename / Delete**. ### Anatomy & formats [#anatomy--formats] Every test case has the same three parts, preconditions, steps, postconditions, whatever the file format: ```md title="tests/example_login_test.md" # Login Flow · Smoke Test ## Preconditions - No user is currently logged in ## Steps 1. Log in using the `login_to_ui` procedure with the QA secrets 2. Wait until the start screen loads ## Postconditions - Test passes if the signed-in indicator is visible ``` ```csv title="tests/example_test_data.csv" Test case ID,Test case name,Precondition,Step number,Step description,Expected result LOGIN_001,Valid login,App is open on the login screen,1,Enter the QA credentials into the login form,Both fields are filled ,,2,Click the Login button,The start screen is visible and the user is signed in ``` ```json title="tests/login_001.json" { "id": "LOGIN_001", "name": "Valid login", "preconditions": ["App is open on the login screen"], "steps": [ { "action": "Enter the QA credentials into the login form", "expected": "Both fields are filled" }, { "action": "Click the Login button", "expected": "The user is signed in" } ] } ``` **Preconditions**: the state the test *assumes*, not work to perform: which screen is open, which user is logged out, which data exists. The agent checks them before the first step and records whether they were met; a test whose preconditions aren't met is reported `SKIPPED` instead of producing a misleading step failure, because it never ran. Shared preparation work belongs in [setup](/docs/writing-tests/setup-teardown-rules), not here. **Steps**: numbered actions, each paired with what to expect. The agent executes them one at a time: perform the action, compare what it observes against the step's expected result, record both with a screenshot. Each step becomes an expected-vs-actual entry in the [run report](/docs/results/run-report), so a step without an expected result is a step the report can't judge. Everything a step can express, conditions, waits, checks, procedure calls, is on [Step constructs](/docs/writing-tests/step-constructs). **Postconditions**: the overall pass criterion, phrased as a checkable end state: "Test passes if …". The agent judges it after the last step, independent of which path it took to get there. The case's reported status is the worst of its steps ([how it is derived](/docs/results/run-report#test-report)). These templates are starting points, not schemas, the agent reads a test the way a tester would, so you're free to choose your own structure. `.md` holds one case per file, `.csv` one case per **row** (data-driven testing), and `.json`, `.txt`, or `.pdf` exports from a test-management tool run as-is. New to Markdown? The [Markdown Guide](https://www.markdownguide.org/basic-syntax/) covers the syntax in ten minutes. ### Target multiple platforms [#target-multiple-platforms] A first line `platforms: desktop, android` marks a case cross-platform, it then runs only on a [Cross-platform profile](/docs/devices/cross-platform), and single-platform tests never run there. The steps name the surface, like a [multi-computer test](/docs/devices/multi-computer) names machines. Cross-platform profiles connect and report status, but runs against them are not supported yet, the case is ready for when they are. ## Use secrets in a test [#use-secrets-in-a-test] Never write credentials into a case, reference [named secrets](/docs/extending/secrets) by name instead. Add the value once under **Extending → Secrets**; the agent sees only the name, and the real value is substituted at execution time, redacted from logs and reports: ```md Sign in with the QA credentials — the secrets named `QA_USERNAME` and `QA_PASSWORD`. ``` ## Run a test case [#run-a-test-case] 1. Pick a connected device in **Run on**. 2. Press **Run**, or right-click any file or folder → **Run on** → a profile. Details in [Running a test](/docs/running-tests). ## Manage a test case [#manage-a-test-case] All of it is plain file operations via right-click, visible in Git like any other change to your [testware](/docs/projects). ### Duplicate [#duplicate] 1. Right-click the file → **Duplicate**. 2. Rename the copy, it's the starting point for a similar case. ### Rename [#rename] 1. Right-click the file → **Rename**. 2. Enter the new name. The file name is the test's name in runs and reports; reports of old runs keep the old name. ### Delete [#delete] 1. Right-click the file → **Delete**. ### Skip [#skip] There is no skip flag: to leave a case out temporarily, run a [test plan](/docs/writing-tests/plans) that doesn't select it, or move the file out of `tests/`. # Organisation & Scope (/docs/writing-tests/organisation) Two rules build the whole structure: **a file is a test case, a folder is a suite**, and suites nest, so the tree mirrors your test structure plan: | Entry | What it is | | ------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `login_test.md` | A single test case, the file name is the test's name in runs and reports | | `user_matrix.csv` | Also test cases, one per row; every [format](/docs/writing-tests#anatomy--formats) sits side by side in the same tree | | `checkout/` | A suite: its cases run together, framed by the suite's own `setup.md` and under its own `rules.md` **in addition to** the outer one. Nest deeper for sub-suites | | `user-management/` | A sibling suite, no steering files of its own, so only the root ones apply. Suites are independent: a failing `checkout/setup.md` never touches it | | `rules.md`, `setup.md`, `teardown.md` | [Steering files](/docs/writing-tests/setup-teardown-rules), allowed at **every** level, each scopes to its folder and everything below | ## Scope [#scope] The steering files reach downward, never sideways. In the tree above, `tests/checkout/pay_with_card.md` runs framed by `tests/setup.md` *and* `tests/checkout/setup.md`, and rules **accumulate**: the root `tests/rules.md` applies to every phase, `tests/checkout/rules.md` adds the suite's own rules on top, but only inside `tests/checkout/`. `tests/user-management/create_user.md` gets only the root files. Setups run top-down, teardowns bottom-up, pressing **Run** on the `checkout` suite: A failing setup blocks exactly the tests below it; sibling suites are unaffected. Any level can be a run target: one file, one suite, or the whole tree, folder structure is also your run granularity. # Test plans (/docs/writing-tests/plans) A test plan is your test selection: which cases run together, and on which device. The classic examples, a fast `smoke` set for every change, the full regression pack for the release, a nightly suite. A plan is a yaml file in `plans/`. You never write it by hand — you check tests and pick a device on the **Test Plans** page and the file is written for you — but it reads (and diffs) like this: ```yaml title="plans/smoke.yaml" version: 1 tests: - tests/login/ - tests/checkout/happy_path.md devices: - id: a1b2c3 ``` * **`tests`**: the selection, in execution order. An entry ending in `/` is a whole folder — it is expanded when the run starts, so a test added to the folder later is included without touching the plan. * **`devices`**: the device the plan runs on, by profile id from `devices.json` (a list in the file, one entry via the app). Only the id is stored — the app shows the device's current name, so renaming a device never touches your plan files. The run contains exactly what the plan lists. Nothing infers the selection, and a plan that selects no tests cannot be run. ## Create a plan [#create-a-plan] 1. Open **Test Plans** and click **New test plan**. A dialog opens. 2. Type a name, for example `smoke`. 3. Check tests in the tree: individual files, a folder to include everything in it, or **All tests** to run the whole project. Files inside a checked folder show as included; uncheck the folder to pick them individually. 4. Pick the device the plan targets. Every profile from the [Devices page](/docs/devices) is offered — the device does not need to be connected while you edit, only when the plan runs. 5. Click **Create plan**. The plan is saved as `plans/smoke.yaml` (the folder is created with the first plan). The page lists each plan with its test count and device; the pencil on a row reopens the dialog. A plan change shows up in review as a small yaml diff, like the rest of your testware. ## Run a plan [#run-a-plan] Press **Run** on the plan. A run starts on the plan's device and executes every selected test — no further input. It appears on the **Runs** page; runs on the same device queue behind each other. The button is disabled while the plan selects no tests or no device, with the reason in the tooltip. If the plan's device is busy (or the two-runs limit is reached), a dialog asks whether to [queue the run](/docs/running-tests#if-it-cant-start-right-away). Two things to know about how a plan run reports: * A selected test the run never reaches is listed as `BROKEN` in the [run report](/docs/results/run-report) — selected means accounted for. * A plan entry whose file was renamed or deleted is skipped with a notice in the conversation log; the run continues with the rest. You can also run a plan headless with [`askui run --plan smoke`](/docs/running-tests/cli) — the CLI runs the selection on the device its flags configure and ignores the plan's device list — or on a [schedule](/docs/running-tests/scheduling), which runs it on the plan's device the same way. ## If a device disappears [#if-a-device-disappears] A plan remembers its device by profile id. When the referenced profile no longer exists, the plan's row says so and the edit dialog shows the orphaned id with a remove button — pick a new device and the plan runs again. ## Older prose plans [#older-prose-plans] Plans used to be free-form Markdown files that an agent interpreted at run time. Those files are no longer runnable and no longer appear on the Test Plans page (they stay on disk until you delete them). Recreate the selection as a test plan — the checkboxes cover what the prose described. # Procedures (/docs/writing-tests/procedures) A procedure is a parameterized, reusable block of steps, the same idea as a keyword in keyword-driven testing. When the login screen changes, you fix the procedure once; every test that calls it picks the fix up. ## Define it [#define-it] The **Procedures** button on the Tests page creates a file in `procedures/` from a template: ```md title="procedures/login_to_ui.md" # Login to the UI # Parameters: [username, password] 1. Enter the value of {username} into the username field 2. Enter the value of {password} into the password field 3. Click the "Login" button 4. The procedure is successful if you are logged in ``` * The **file name** is the name tests call: `login_to_ui`. * The `# Parameters: [...]` line declares the inputs; steps reference them as `{username}`, `{password}`. * The last step states when the procedure counts as successful. ## Use it [#use-it] Call it as a single step from any test, or from a suite's [setup](/docs/writing-tests/setup-teardown-rules), in plain language: name the procedure and say what fills its parameters; a [secret](/docs/extending/secrets) is passed by name: ```md 1. Log in using the `login_to_ui` procedure with the QA username and password secrets. 2. Wait until the start screen loads ``` What belongs in a procedure versus a test step: [What goes where](/docs/best-practices/prompting-best-practices). # Setup, Teardown & Rules (/docs/writing-tests/setup-teardown-rules) Every suite folder can carry three steering files. Their [scope chains per folder level](/docs/writing-tests/organisation#scope); each quick-create seeds a starter template, adjust it to your application, keep the shape. ## Setup [#setup] The suite's entry criteria: open the application, log in, reset data. Runs once, before any case in its folder, a failing setup records the suite's cases as broken instead of executing them against a bad state. **Create it:** right-click the folder → **New setup.md**. ```md title="setup.md" ## Setup Steps 1. Open the application under test and wait until it is fully visible. 2. Log in using the `login_to_ui` procedure with the QA username and password secrets. 3. Verify the start screen has loaded before any test starts. ``` ## Teardown [#teardown] The suite's cleanup: sign out, delete created records. Runs last, after every case in its folder, even after failures. **Create it:** right-click the folder → **New teardown.md**. ```md title="teardown.md" ## Teardown Steps 1. Close the application under test. 2. Clear any temporary files created during testing. ``` ## Rules [#rules] Standing instructions that apply to every phase in the folder, not a test, context: interaction style, error handling, things never to do ("never change the language setting", "treat error dialogs as failures"). **Create it:** right-click the folder → **New rules.md**. ```md title="rules.md" # Execution rules for this folder ## Interaction - Use the desktop computer to execute these tests. ## Error handling - On an infrastructure error (connection lost, session expired): mark the testcase BROKEN, write the report, abort. - On an application error: mark the testcase FAILED and abort. ``` # Step constructs (/docs/writing-tests/step-constructs) Steps are plain language, not syntax. Imagine handing the test to a human colleague **without domain knowledge about your application**: whatever they would need from you to execute it, which element, what to expect, what to do if a dialog appears, what never to touch, is exactly what the agent needs. If that colleague could run the step from your wording alone, so can the agent. These constructs are the vocabulary that works, each example is from a real seeded test. They apply wherever steps live: in [test cases](/docs/writing-tests), in [setup and teardown](/docs/writing-tests/setup-teardown-rules), and inside [procedure](/docs/writing-tests/procedures) bodies. ## Actions [#actions] Describe intent; the agent works out the clicks and keystrokes: ```md 1. Open the Start menu, type `Notepad`, and press Enter. ``` ## Locators [#locators] Describe a UI element the way you'd point it out to a colleague, by any visible feature, combined freely: | Kind | Example | | ------------ | ----------------------------------------------------------------------------------- | | **Text** | Click the button labeled **"Save"** | | **Icon** | Click the **gear icon** in the toolbar | | **Color** | Click the **green** Connect button | | **Location** | The search field **at the top right** | | **Relation** | The Delete button **next to** the second list entry, the field **below** "Username" | The more distinguishing features, the more reliable the step, "click the button" fails on a screen with ten buttons; "the green Connect button on the Windows Desktop card" doesn't. Phrasing guidance: [Writing good tests](/docs/best-practices/writing-good-tests). ## Expectations [#expectations] Pair every action with what must be true afterwards, that pair becomes the step's expected-vs-actual entry in the [report](/docs/results/run-report): ```md 2. Click the Login button — the start screen is visible and the user is signed in. ``` ## Verifications [#verifications] A step can be pure checking: ```md 3. Verify the response status is 200 and the body contains "Example Domain". 4. Check whether `tests/example_test_data.csv` exists (expect yes). ``` ## Conditions [#conditions] Because the agent [reads the screen before every step](/docs/concepts/test-automation-harness#why-execution-differs), steps can branch on what is actually there: ```md 5. If the result says more rows are available, call it again with the suggested offset — otherwise continue. 6. If a cookie banner appears, dismiss it before continuing. ``` Bigger decisions read best as sub-steps, one branch per letter, each with its own expected result: ```md 7. Check whether a window named "AskUI Desktop" is already open: a. If yes: bring it to the front — the app window is focused. b. If no: open the Start menu, search for "AskUI Desktop" and start it — the app window opens within a minute. ``` ## Optional [#optional] A step that only applies in some situations, and whose absence is **not** a failure. Say what to do when the situation doesn't occur, so the report shows SKIPPED instead of FAILED: ```md 8. If the tutorial overlay is shown on first start, dismiss it — otherwise mark this step SKIPPED and continue. 9. Suppress pop-ups throughout: whenever a "Rate this app" pop-up appears, close it and continue — if none appears, that's fine. ``` ## Waits [#waits] Normally you don't need waits, the agent sees the screen and won't act on a half-loaded page. When it gets impatient anyway, tell it **what to look for**, not how long to wait: ```md 10. Wait until the start screen loads. ``` A fixed duration is the last resort, for settling you can't describe: ```md 11. Wait 3 seconds for the page to settle. ``` ## Timeouts [#timeouts] The counterpart of a wait: bound how long the agent keeps trying, and say what happens when time is up, otherwise a hanging application under test turns into an endlessly patient agent: ```md 12. Start the export. If the success message does not appear within about 2 minutes, mark the step FAILED and continue with the next test. ``` Timeouts are approximate, the agent judges elapsed time between screenshots, it doesn't run a stopwatch. For hard behavior on errors and hangs that applies to every test, use [rules.md](/docs/writing-tests/setup-teardown-rules#rules). ## Raising errors [#raising-errors] Tell the agent to fail **actively** when the application doesn't react as expected, an agent that helpfully works around a defect masks exactly what the test exists to find: ```md 13. Open the customer record ACME-42. If an error dialog appears instead, mark the test FAILED and stop — do NOT retry or work around it. ``` Use the [status vocabulary](/docs/results/run-report#test-report): **FAILED** when the application misbehaves, **BROKEN** for infrastructure problems (connection lost, session expired). The seeded [rules.md](/docs/writing-tests/setup-teardown-rules#rules) already sets this policy for every test, a step-level instruction sharpens it for a specific known risk. ## Dynamic values [#dynamic-values] Values that only exist at run time, an ID the application generates, a number shown on screen, can be noted in one step and referenced in a later one; the agent carries what it saw through the whole test: ```md 14. Create a new user — note the user ID shown in the confirmation dialog. 15. Open the user list and click the entry with the user ID noted in step 12. ``` The seeded cross-platform example uses the same pattern across devices: read the most recent order number on the desktop, then search for exactly that number on the phone. ## Iteration [#iteration] A step can repeat over what's on screen, "for every …" across lists, tabs, table rows: ```md 16. For every tab in the Settings dialog: open it and verify it loads without an error message. ``` The loop body can hold several sub-steps, the same lettered pattern as conditions. Keep the set bounded: ```md 17. For the first 3 orders in the list: a. Open the order — the detail view shows its order number. b. Verify the order contains at least one line item. c. Add the comment "checked by QA" and save — the comment appears in the order history. d. Return to the order list — it shows the same rows as before. ``` The agent has to keep count on a screen that changes with every action, long or unbounded iterations drift. Keep the set small and verifiable ("all 4 tabs"), and when the elements are known up front, prefer one [data-driven CSV case per element](/docs/writing-tests#anatomy--formats) instead of a loop. This is a limitation of today's models and improves with every model generation, expect the caution to relax over time. ## Exclusions [#exclusions] Say explicitly what must **not** happen, the agent honors prohibitions like a tester honors a warning in the test spec: ```md 18. Click **Submit** exactly once — do NOT press it again, even if the page seems to hang. 19. Do NOT dismiss the license dialog; the test verifies its content first. ``` For prohibitions that apply to every test in a folder ("never change the language setting", "never sign out"), put them in [rules.md](/docs/writing-tests/setup-teardown-rules#rules) instead of repeating them per step. ## Comments [#comments] HTML comments are notes for maintainers, why a step exists, a ticket reference, a review hint. They are not steps: ```md ``` The agent reads comments as background, not instructions, but it does read them, so keep them truthful: a stale or misleading comment can steer execution the same way a misleading remark steers a colleague. The seeded templates use comments exactly this way. ## Procedure calls [#procedure-calls] One step runs a whole [procedure](/docs/writing-tests/procedures), name it and say what fills its parameters: ```md 20. Log in using the `login_to_ui` procedure with the QA username and password secrets. ``` ## Secrets [#secrets] Reference credentials by name, never by value, [secrets](/docs/extending/secrets): ```md 21. Sign in with the QA credentials — the secrets named `QA_USERNAME` and `QA_PASSWORD`. ``` ## Tool invocations [#tool-invocations] Steps can name anything the [toolbox](/docs/extending/tools) allows, HTTP requests, SQL statements, shell scripts, QR scanning. Describing the action is usually enough; when two tools could match, name the tool explicitly: ```md 22. Perform an HTTP GET request to `https://example.com`. 23. Scan the QR code from the screen — the decoded text contains "wikipedia". 24. Use the `sql_database` tool to run the statement `test_data`. ``` Tool names are listed under **Extending → Tool Store**, each card carries the name the agent knows it by. A [custom tool](/docs/extending/custom-tools)'s name is the `name:` in its C# file. ## Targeting machines and platforms [#targeting-machines-and-platforms] In a [multi-computer](/docs/devices/multi-computer) test, name the machine per step; a `platforms:` first line marks a [cross-platform case](/docs/writing-tests#target-multiple-platforms): ```md 25. On the **server**, start the message service. 26. On the **client**, send a message to the server. ``` ## Beyond steps [#beyond-steps] Some things don't belong in any step, they're the manual tester's tribal knowledge, and they have their own homes. **Real-world disruptions.** Some failures are random and expected: the application hangs, the printer is empty, a product is out of stock. A manual tester doesn't fail the test, they restart the app, ignore the printer, refill the quantity, and carry on. Give the agent the same recoveries in [rules.md](/docs/writing-tests/setup-teardown-rules#rules), at the [scope](/docs/writing-tests/organisation#scope) that needs them: ```md title="tests/rules.md" ## Recoveries - If the application hangs, restart it and repeat the current step once. - If the printer reports empty, ignore the print result and continue. - If a product is out of stock, set its quantity to 10 in inventory first. ``` **The agent operates the UI differently than you wish.** It uses keyboard shortcuts where your users click menus, types numbers instead of using the on-screen numpad, keeps misreading the same part of your UI. That's not a step problem either, teach it the house style in [rules.md](/docs/writing-tests/setup-teardown-rules#rules) or, when it's about how your application works, in the [UI information file](/docs/writing-tests/agents-and-prompts#ui-information): ```md title="tests/ui.md" - Enter numbers via the on-screen numpad with mouse clicks — never type them with the keyboard. - Do not use keyboard shortcuts; this application is tested through its menus, like our users work. ``` What belongs where in detail: [What goes where](/docs/best-practices/prompting-best-practices). # CI pipeline examples (/docs/agentos/deployment/ci-examples) CI execution needs the AskUI CLI, which is not publicly available yet. This page describes the setup you will use once it ships, so you can plan for it. These examples show how to run `askui run` in the most common CI environments. All examples assume: * The AskUI Desktop installer has been run on the CI machine (or a self-hosted runner with AskUI installed), since the CLI ships with the Desktop installer. * `ASKUI_WORKSPACE_ID` and `ASKUI_TOKEN` are stored as CI secrets. * Your test project is checked into the repository. Desktop automation on Windows requires AgentOS running as a service. Install it via the AskUI Desktop installer on your runner before running the pipeline. See [CI/CD integration](/docs/agentos/deployment/ci) for setup details. ## GitHub Actions [#github-actions] ```yaml # .github/workflows/askui.yml name: AskUI Tests on: push: branches: [main] pull_request: jobs: test: runs-on: windows-latest # or macos-latest for macOS desktop / web steps: - name: Checkout uses: actions/checkout@v4 - name: Run AskUI tests env: ASKUI_WORKSPACE_ID: ${{ secrets.ASKUI_WORKSPACE_ID }} ASKUI_TOKEN: ${{ secrets.ASKUI_TOKEN }} run: askui run - name: Upload run reports if: always() uses: actions/upload-artifact@v4 with: name: askui-reports path: agent_workspace/ ``` ## GitLab CI [#gitlab-ci] ```yaml # .gitlab-ci.yml askui-tests: stage: test tags: - windows # use a self-hosted runner with AskUI installed variables: ASKUI_WORKSPACE_ID: $ASKUI_WORKSPACE_ID ASKUI_TOKEN: $ASKUI_TOKEN script: - askui run artifacts: when: always paths: - agent_workspace/ expire_in: 7 days ``` ## Azure DevOps [#azure-devops] ```yaml # azure-pipelines.yml trigger: - main pool: vmImage: windows-latest # or a self-hosted agent with AskUI installed steps: - checkout: self - script: askui run displayName: Run AskUI tests env: ASKUI_WORKSPACE_ID: $(ASKUI_WORKSPACE_ID) ASKUI_TOKEN: $(ASKUI_TOKEN) - task: PublishBuildArtifacts@1 condition: always() inputs: pathToPublish: agent_workspace/ artifactName: askui-reports ``` ## Connecting to a remote AgentOS [#connecting-to-a-remote-agentos] If your CI runner is not the machine being automated (e.g. a Linux runner automating a separate Windows VM), disable AgentOS autostart and point the CLI at the remote instance: ```bash export ASKUI_CONTROLLER_CLIENT_SERVER_AUTOSTART=false export ASKUI_CONTROLLER_CLIENT_SERVER_ADDRESS=:26000 askui run ``` See [Remote Windows VM](/docs/agentos/deployment/ci#remote-windows-vm) for the full setup. ## Storing credentials [#storing-credentials] Never hardcode `ASKUI_WORKSPACE_ID` or `ASKUI_TOKEN` in your pipeline files. Use your CI platform's secret store: | Platform | Where to add secrets | | -------------- | ----------------------------------------------------- | | GitHub Actions | Repository Settings → Secrets and variables → Actions | | GitLab CI | Settings → CI/CD → Variables | | Azure DevOps | Pipeline → Edit → Variables | Create a dedicated access token per pipeline so you can rotate or revoke individual pipelines without affecting others. See [Workspaces](/docs/account-billing/workspaces). # CI/CD Integration (/docs/agentos/deployment/ci) You're running agents unattended in a pipeline or on remote infrastructure. The install method depends on whether you need OS service capabilities (RDP resilience, SYSTEM privileges) or standalone mode is sufficient. ## Windows VM [#windows-vm] Run automation directly on a Windows VM that is part of your CI/CD pipeline. AgentOS runs as an OS service with SYSTEM privileges. **When to use:** Your CI runner *is* the Windows machine you want to automate. **Install:** [Service](/docs/agentos/installation/service) on the VM. AskUI Desktop or the CLI and AgentOS both run on the same VM. The OS service ensures automation continues even if no user is logged in or the RDP session disconnects. ## Remote Windows VM [#remote-windows-vm] Automate a Windows VM from a separate CI runner. AskUI Desktop or the CLI runs on the CI runner and connects to AgentOS on the remote VM over the network. **When to use:** Your CI runner is Linux/macOS but you need to automate a Windows desktop. Or your pipeline orchestrates work across multiple machines. **Install:** [Service](/docs/agentos/installation/service) on the remote VM. During installation, set **Connection Scope** to **Public (0.0.0.0)**. The default Private (127.0.0.1) only accepts local connections and will prevent the client from reaching AgentOS over the network. The client connects to AgentOS over the network via gRPC. The OS service on the remote VM handles desktop control independently, RDP disconnects, logon screens, and headless operation are all supported.

Connecting to a remote AgentOS
By default, AskUI Desktop and the CLI start a local AgentOS instance. To connect to a remote VM instead, disable autostart and point the client to the remote address. **Disable local autostart** Set `ASKUI_CONTROLLER_CLIENT_SERVER_AUTOSTART` to `false` so the client doesn't start a local AgentOS instance. ```powershell $env:ASKUI_CONTROLLER_CLIENT_SERVER_AUTOSTART="false" ``` ```bash export ASKUI_CONTROLLER_CLIENT_SERVER_AUTOSTART=false ``` **Set the remote address** Point `ASKUI_CONTROLLER_CLIENT_SERVER_ADDRESS` to the AgentOS service on the remote VM. Replace `192.168.1.100` with your VM's IP address. ```powershell $env:ASKUI_CONTROLLER_CLIENT_SERVER_ADDRESS="192.168.1.100:26000" ``` ```bash export ASKUI_CONTROLLER_CLIENT_SERVER_ADDRESS=192.168.1.100:26000 ``` **Verify the connection** Run your agent. The client should connect to the remote AgentOS instance instead of starting a local one. Check that the remote VM's port `26000` is reachable from your CI runner.
## macOS / Linux CI Runner [#macos--linux-ci-runner] Run automation on a macOS or Linux CI runner. AgentOS runs in standalone mode. **When to use:** Your CI pipeline runs on macOS or Linux and you want to automate the desktop on that same runner. **Install:** [AskUI Desktop installer](/docs/get-started/install-desktop) (includes AgentOS). A standalone installer for CI use is coming soon. Same as local development, AgentOS runs in standalone mode alongside your test code. Ensure the CI runner has a display (real or virtual) available. ## Mobile Device in CI [#mobile-device-in-ci] Automate Android or iOS devices connected to your CI runner via USB. **When to use:** Mobile testing in your pipeline, the device is physically connected to the CI runner. **Install:** [AskUI Desktop installer](/docs/get-started/install-desktop) (includes AgentOS) on the CI runner. A standalone installer is coming soon. AgentOS runs on the CI runner and communicates with the connected device over USB. The CI runner needs physical USB access to the device (or a USB-over-network solution). ### Android: Setting up ADB [#android-setting-up-adb] ADB (Android Debug Bridge) must be available on your PATH. ADB works with both physical devices and emulators. See Google's guide on [running apps on the Android Emulator](https://developer.android.com/studio/run/emulator) to get started with emulators. **Download SDK Platform Tools** Download from [developer.android.com](https://developer.android.com/tools/releases/platform-tools) and unzip to a folder (e.g. `C:\platform-tools` or `~/platform-tools`). **Add to PATH** Add the unzipped folder to your **User** or **System PATH** environment variable. **Verify** Open a new terminal and run: ```bash adb version ``` You should see the ADB version output. ### iOS: Setting up IDB (macOS only, experimental) [#ios-setting-up-idb-macos-only-experimental] iOS automation requires macOS with Xcode and the Facebook IDB companion. IDB currently only works with **iOS Simulators**, not physical devices. This feature is **experimental**. See Apple's guide on [running your app in Simulator](https://developer.apple.com/documentation/xcode/running-your-app-in-simulator-or-on-a-device) to get started. **Install Xcode with iOS Simulators** Install [Xcode](https://developer.apple.com/xcode/) from the App Store and configure iOS Simulators. Verify they are visible: ```bash xcrun xctrace list devices ``` **Install IDB companion** ```bash brew tap facebook/fb brew install idb-companion ``` **Verify** Open a new terminal and run: ```bash idb_companion --list 1 ``` You should see your available simulators or devices listed. # Local Development (/docs/agentos/deployment/local) You're building and testing agents on your own machine. In all local scenarios, AgentOS is installed via the AskUI installer, and AskUI Desktop and the CLI connect to the local AgentOS instance over gRPC. ## Desktop [#desktop] Automate the desktop on your own Windows, macOS, or Linux machine. AgentOS runs in standalone mode alongside your agent code. **When to use:** Day-to-day development, debugging, and interactive testing. **Install:** [AskUI Desktop installer](/docs/get-started/install-desktop) (includes AgentOS) → [Standalone](/docs/agentos/installation/standalone) Your agent code and AgentOS run on the same machine. AskUI Desktop and the CLI send commands via gRPC, and AgentOS translates them into OS-level actions, screenshots, keyboard input, mouse control. ## Mobile Device [#mobile-device] Automate an Android or iOS device connected to your machine via USB. **When to use:** Mobile app testing, device interaction during development. **Install:** [AskUI Desktop installer](/docs/get-started/install-desktop) (includes AgentOS) → [Standalone](/docs/agentos/installation/standalone) AgentOS acts as a bridge: it receives commands via gRPC and forwards them to the connected device using ADB (Android) or IDB (iOS). The device stays connected via USB. You only need one of ADB or IDB depending on your target device, both are shown for completeness. ### Android: Setting up ADB [#android-setting-up-adb] ADB (Android Debug Bridge) must be available on your PATH. ADB works with both physical devices and emulators. See Google's guide on [running apps on the Android Emulator](https://developer.android.com/studio/run/emulator) to get started with emulators. **Download SDK Platform Tools** Download from [developer.android.com](https://developer.android.com/tools/releases/platform-tools) and unzip to a folder (e.g. `C:\platform-tools` or `~/platform-tools`). **Add to PATH** Add the unzipped folder to your **User** or **System PATH** environment variable. **Verify** Open a new terminal and run: ```bash adb version ``` You should see the ADB version output. ### iOS: Setting up IDB (macOS only, experimental) [#ios-setting-up-idb-macos-only-experimental] iOS automation requires macOS with Xcode and the Facebook IDB companion. IDB currently only works with **iOS Simulators**, not physical devices. This feature is **experimental**. See Apple's guide on [running your app in Simulator](https://developer.apple.com/documentation/xcode/running-your-app-in-simulator-or-on-a-device) to get started. **Install Xcode with iOS Simulators** Install [Xcode](https://developer.apple.com/xcode/) from the App Store and configure iOS Simulators. Verify they are visible: ```bash xcrun xctrace list devices ``` **Install IDB companion** ```bash brew tap facebook/fb brew install idb-companion ``` **Verify** Open a new terminal and run: ```bash idb_companion --list 1 ``` You should see your available simulators or devices listed. ## KVM (External Hardware) [#kvm-external-hardware] Control a target device through physical hardware connections, keyboard/mouse via USB or Bluetooth, screen capture via HDMI. **When to use:** The target device can't have software installed on it (locked-down environments, embedded systems, kiosks). **Install:** [AskUI Desktop installer](/docs/get-started/install-desktop) (includes AgentOS) → [Standalone](/docs/agentos/installation/standalone) This is [Companion Mode](/docs/agentos/understanding/control-modes#companion-mode): AgentOS simulates keyboard and mouse input over USB or Bluetooth HID, and captures the target's screen via an HDMI-to-USB capture device. No software installation on the target is required. # Multi-Device Setups (/docs/agentos/deployment/multi-device) You need to automate across multiple targets, desktops, VMs, mobile devices, or hardware. In multi-device setups, AskUI Desktop and the CLI connect to **each AgentOS instance separately** via gRPC. There is no AgentOS-to-AgentOS communication. ## Windows + Mobile Device [#windows--mobile-device] Automate a Windows desktop and a mobile device from the same pipeline. **When to use:** Cross-platform testing, e.g., a web app on Windows and its companion mobile app. The client manages two independent gRPC connections: one to the AgentOS OS Service on the Windows VM, and one to a local AgentOS instance that controls the mobile device. ## Multiple Windows VMs [#multiple-windows-vms] Scale desktop automation across several Windows VMs in parallel. **When to use:** Running the same tests across different Windows configurations, or distributing a large test suite across machines for faster execution. Each VM runs its own AgentOS OS Service. The client connects to all of them independently. Your pipeline code decides which commands go to which VM. ## Multiple Mobile Devices [#multiple-mobile-devices] Automate several Android (or iOS) devices connected to the same machine. **When to use:** Testing across different device models, screen sizes, or OS versions in parallel. Each device gets its own AgentOS instance. The client connects to each one over gRPC and routes commands to the right device. ## Desktop + KVM Device [#desktop--kvm-device] Combine software-based desktop control with hardware-based control of an external device. **When to use:** Testing interactions between a desktop application and a physical device that can't have software installed (e.g., an embedded system, kiosk, or industrial controller). Two AgentOS instances run on the same machine, each in a different [control mode](/docs/agentos/understanding/control-modes): one controls the local desktop (Host Mode), the other controls the external device through hardware (Companion Mode). The client connects to both independently. # Enterprise Features (/docs/agentos/reference/enterprise) AgentOS is designed for enterprise environments that require security controls, audit capabilities, and compliance features. ## Security & Compliance [#security--compliance] | Feature | Status | Description | | ------------------------ | --------- | ---------------------------------------------------------------------------------------- | | Signed binaries | Available | Code-signed installers and binaries, no SmartScreen or Execution Policy bypasses needed. | | Application whitelisting | Planned | Restrict which applications AgentOS can interact with. | | Filesystem whitelisting | Planned | Restrict filesystem paths AgentOS can access. | | Audit logs | Planned | Detailed logs of all actions taken by AgentOS for compliance and forensics. | | Application logs | Planned | Structured application-level logging for monitoring. | | Network scanning | Planned | Visibility into network activity originating from AgentOS. | ## Support [#support] | Feature | Status | Description | | ----------------- | ------- | ------------------------------------------------------- | | Dedicated support | Planned | Priority support channel for enterprise customers. | | Vulnerability SLA | Planned | Guaranteed response times for security vulnerabilities. | ## Connectivity [#connectivity] | Feature | Status | Description | | -------------- | ------- | ---------------------------------------------------------- | | P2P connection | Planned | Direct peer-to-peer connections between AgentOS instances. | | Video stream | Planned | Real-time video streaming of the controlled desktop. | | File stream | Planned | File transfer including log file streaming. | ## Industry Protocols [#industry-protocols] Integration with industrial communication standards for factory automation, IoT, and embedded testing. | Protocol | Status | Description | | ------------------------ | ------- | ----------------------------------------------------- | | CAN bus | Planned | Automotive and industrial controller area network. | | OPC | Planned | Classic OPC for industrial data exchange. | | OPC UA | Planned | Modern OPC Unified Architecture. | | MQTT | Planned | Lightweight messaging for IoT devices. | | Zigbee | Planned | Low-power wireless for smart devices. | | Serial (RS-232 / RS-485) | Planned | Serial communication for legacy and embedded devices. | ## Test Bench (Extended Testing Devices) [#test-bench-extended-testing-devices] Physical hardware integration for automated test setups. | Device | Status | Description | | ------------------------ | ------- | ----------------------------------------------- | | Relay control (Ethernet) | Planned | Switch power or signals via Ethernet relays. | | Relay control (USB) | Planned | Switch power or signals via USB relays. | | SwitchBot / Fingerbot | Planned | Physical button pressing via robotic actuators. | | Bluetooth dongle | Planned | Bluetooth connectivity for test devices. | | Wi-Fi dongle | Planned | Wi-Fi connectivity for test devices. | | Workbench integration | Planned | Full test bench orchestration. | # System Requirements (/docs/agentos/reference/system-requirements) ## Operating System Requirements [#operating-system-requirements] * **Windows**: Windows 10 version 1511+ (64-bit), Windows 11 (64-bit), Windows Server 2019/2022/2025 (64-bit) * **macOS**: macOS 10.15 (Catalina) or newer * **Linux**: Ubuntu 18.04 or newer ### Supported Windows Versions [#supported-windows-versions] #### Windows 10 [#windows-10] | Version | Name | Supported | | ------- | -------------------- | --------- | | 1511 | November Update | x64 | | 1607 | Anniversary Update | x64 | | 1703 | Creators Update | x64 | | 1709 | Fall Creators Update | x64 | | 1803 | April 2018 Update | x64 | | 1809 | October 2018 Update | x64 | | 1903 | May 2019 Update | x64 | | 1909 | November 2019 Update | x64 | | 2004 | May 2020 Update | x64 | | 20H2 | October 2020 Update | x64 | | 21H1 | May 2021 Update | x64 | | 21H2 | November 2021 Update | x64 | | 22H2 | October 2022 Update | x64 | #### Windows 11 [#windows-11] | Version | Name | Supported | | ------- | ---------------- | --------- | | 21H2 | Original Release | x64 | | 22H2 | 2022 Update | x64 | | 23H2 | 2023 Update | x64 | | 24H2 | 2024 Update | x64 | | 25H2 | 2025 Update | x64 | #### Windows Server [#windows-server] | Version | Channel | Supported | | ------- | -------------- | --------- | | 2019 | LTSC | x64 | | 2022 | LTSC | x64 | | 2025 | LTSC | x64 | | 23H2 | Annual Channel | x64 | ## Hardware Requirements [#hardware-requirements] ### Minimum Specifications [#minimum-specifications] * **Memory**: 8 GiB RAM * **CPU**: 4 CPU cores ### Companion Mode (Additional) [#companion-mode-additional] * **USB port** for HID input or device connection (ADB/IDB) * **Bluetooth** adapter for wireless HID input (optional) * **HDMI capture device** (USB) for screen capture (optional) ## Software Requirements [#software-requirements] AgentOS is installed via the AskUI installer and has no separate software prerequisites. # Capabilities (/docs/agentos/understanding/capabilities) AgentOS provides a set of capabilities for controlling machines. What's available depends on the [control mode](/docs/agentos/understanding/control-modes) and [runtime mode](/docs/agentos/understanding/runtime-modes) you use. ## Desktop Control [#desktop-control] | Capability | Host Mode | Companion Mode | Description | | --------------------------- | --------- | ----------------------------------- | ----------------------------------------------- | | Screenshots | Available | Available (via HDMI capture) | Capture the current screen state. | | GPU-accelerated screenshots | Available | , | Take screenshots on GPU for better performance. | | Keyboard input | Available | Available (via USB / Bluetooth HID) | Simulate key presses and combinations. | | Real Unicode typing | Available | , | Native Unicode input (not clipboard-based). | | Mouse input | Available | Available (via USB / Bluetooth HID) | Click, move, drag, and scroll. | | Multi-display support | Available | , | Interact with multiple monitors. | | Window management | Available | , | Select, resize, move, and focus windows. | | Process management | Available | , | Start, stop, and monitor OS processes. | | Automation recovery | Available | , | Recover automatically from display changes. | | CLI | Available | , | Enhanced command-line interface. | ## Windows Service (OS Service Mode Only) [#windows-service-os-service-mode-only] These capabilities require AgentOS to be installed as a [Windows service](/docs/agentos/installation/service). | Capability | Status | Description | | ----------------------------- | --------- | ----------------------------------------------------- | | SYSTEM-level service | Available | Run as a Windows service with SYSTEM privileges. | | RDP disconnect recovery | Available | Keep automation running when RDP sessions disconnect. | | Login screen automation | Available | Interact with the Windows logon screen. | | Secure Attention Sequence | Available | Send CTRL+ALT+DEL programmatically. | | Background session automation | Available | Automate sessions that are not in the foreground. | | Multi-user session control | Planned | Control multiple user sessions on the same machine. | | Virtual display | Planned | Create virtual displays for headless environments. | ## Mobile Devices [#mobile-devices] | Capability | Status | Description | | ------------- | ------------ | ------------------------------------ | | ADB (Android) | Available | Control Android devices via USB. | | IDB (iOS) | Experimental | Control iOS Simulators (macOS only). | ## Hardware (Companion Mode Only) [#hardware-companion-mode-only] | Capability | Status | Description | | --------------------------- | --------- | ------------------------------------------------------- | | HID over USB | Available | Simulate keyboard and mouse via USB connection. | | HID over Bluetooth | Available | Simulate keyboard and mouse via Bluetooth. | | Screenshot via media stream | Available | Capture screen output via HDMI-to-USB dongle or webcam. | | Raspberry Pi support | Planned | Use a Raspberry Pi as the companion device. | ## Platform Support [#platform-support] | Platform | Host Mode | Companion Mode | | -------------------- | --------- | ----------------------------------- | | Windows 10 22H2+ | Available | , | | Windows 11 | Available | , | | Windows Server 2019+ | Available | , | | Windows 10 \< 22H2 | Available | , | | macOS | Available | , | | Linux | Available | , | | Android | , | Available (ADB) | | iOS | , | Experimental (IDB, Simulators only) | ## Future [#future] | Capability | Status | Description | | ---------------------- | ------- | --------------------------------------------------- | | C# API | Planned | Native C# interface for .NET integrations. | | Future automation APIs | Planned | Integration with emerging OS automation frameworks. | # Concepts (/docs/agentos/understanding/concepts) AgentOS has three core concepts that determine what it can do and how it connects to a target machine. While they involve OS-level services, kernel drivers, and hardware interfaces, **you don't need to understand these internals to use AgentOS**, the installer handles the complexity. AgentOS is **platform independent**, Windows, macOS, Linux, Android, and iOS Simulators (experimental), so the same configuration works across operating systems. ## Control modes [#control-modes] Control modes define **how AgentOS connects to the target machine**: * **Host Mode**: AgentOS runs directly on the target as software, using OS-level APIs for screenshots, input, and window management. The most common setup. * **Companion Mode**: AgentOS runs on a separate device and controls the target through hardware (USB, Bluetooth, HDMI capture) or device bridges (ADB, IDB). No software installation on the target is needed. [Learn more about Control Modes →](/docs/agentos/understanding/control-modes) ## Runtime modes [#runtime-modes] Runtime modes define **how AgentOS runs on the machine it's installed on**: * **Standalone**: runs as a regular process in the user's session. Best for local development and testing. * **OS Service**: runs as a system service with elevated privileges. Best for CI/CD, headless VMs, and enterprise deployments. Runtime mode is independent of control mode, both Host and Companion mode can run Standalone or as an OS Service. Higher privileges unlock additional capabilities such as RDP resilience and access to the Windows logon screen. [Learn more about Runtime Modes →](/docs/agentos/understanding/runtime-modes) ## Capabilities [#capabilities] Both control modes expose the same **capabilities** to the agent: screen reading (screenshots, pixel and element access), input simulation (mouse, keyboard, gestures), and window/process management. The agent uses these as building blocks regardless of how AgentOS is connected to the target. [Learn more about Capabilities →](/docs/agentos/understanding/capabilities) AgentOS is the runtime. You drive it through [AskUI Desktop](/docs/get-started/install-desktop) and the [CLI](/docs/running-tests/cli), no separate SDK required. # Control Modes (/docs/agentos/understanding/control-modes) AgentOS operates in two control modes that determine *how* it connects to and controls a target machine. Both modes expose the same core concepts (screenshots, input, automation), but the connection method differs. ## Host Mode [#host-mode] AgentOS runs directly **on the target machine** as software. It controls the desktop via OS-level APIs. * Agent code and AgentOS run on the same machine (or connect over the network). * Full access to OS-level features: window management, process control, multi-display support. * Can run as a [Windows service](/docs/agentos/installation/service) for SYSTEM privileges, RDP resilience, and logon screen control. * Supports Windows, macOS, and Linux. **Best for:** Desktop automation, CI/CD pipelines, enterprise VMs. ## Companion Mode [#companion-mode] AgentOS runs on an **external device** and controls the target through hardware interfaces. No software needs to be installed on the target. * Input is sent via USB or Bluetooth HID (keyboard and mouse emulation). * Screen capture happens via HDMI-to-USB capture devices. * Mobile devices are controlled via ADB (Android) or IDB (iOS, planned). **Best for:** Locked-down environments, embedded systems, kiosks, mobile devices, and any target where you cannot install software. ## Comparison [#comparison] | | Host Mode | Companion Mode | | ------------------------ | --------------------- | -------------------------------------------------------------------- | | **Where AgentOS runs** | On the target machine | On an external device | | **Connection to target** | OS-level APIs | Hardware (USB, Bluetooth, HDMI) or device bridge (ADB/IDB) | | **Software on target** | Required | Not required | | **OS service support** | Yes (Windows) | No | | **Platform support** | Windows, macOS, Linux | Any device with USB/HDMI/Bluetooth; Android (ADB), iOS (IDB planned) | For a detailed breakdown of what each mode can do, see [Capabilities](/docs/agentos/understanding/capabilities). ## What's Next? [#whats-next] # Runtime Modes (/docs/agentos/understanding/runtime-modes) AgentOS can run in two modes. Choose based on your use case. ## Standalone [#standalone] AgentOS runs as a regular process in the current user's session. **Best for:** Local development, interactive desktop use, and manual testing. AgentOS is installed via the AskUI installer. A dedicated standalone installer (without AskUI Desktop) is coming soon. For now, standalone AgentOS is installed as part of the [AskUI Desktop installer](/docs/get-started/install-desktop). A separate standalone installer is in progress. Once installed, AskUI Desktop and the CLI use the local AgentOS instance automatically. ## OS Service [#os-service] AgentOS runs as a Windows system service with SYSTEM privileges. **Best for:** Enterprise deployments, CI/CD pipelines, headless VMs, and scenarios where RDP sessions may disconnect. The OS service requires the [Windows installer](/docs/agentos/installation/service). Once installed, AskUI Desktop and the CLI automatically use the service instead of standalone mode. ## Feature Comparison [#feature-comparison] | Feature | Standalone | OS Service | | ----------------------------- | ---------------------------------------- | ------------------------------- | | **Primary use** | Local dev / desktop | Enterprise / CI/CD | | **CI/CD ready (Windows)** | No (requires OS service) | Yes (unattended/headless) | | **CI/CD ready (macOS/Linux)** | Yes | , | | **RDP resilience** | No (session locks on disconnect) | Yes (session transfer) | | **Logon screen control** | No | Yes | | **Send CTRL+ALT+DEL** | No | Yes (Secure Attention Sequence) | | **Privileges** | Current user | SYSTEM | | **Install method** | AskUI installer (standalone coming soon) | Windows installer | **Not sure which to pick?** Start with **standalone** for development. Move to the **OS service** when you need CI/CD, headless VMs, or RDP resilience. # OS Service (exe) (/docs/agentos/installation/service) AgentOS is proprietary software, **free for non-commercial use**. For commercial deployments, a license is required. See [pricing](https://www.askui.com/enterprise) for details. ## Prerequisites [#prerequisites] * **Windows** (Windows 10, Windows 11, Windows Server 2019+), see [System Requirements](/docs/agentos/reference/system-requirements) for supported versions. * **Administrator rights** To automate the login/lock screen (Secure Attention Sequence), a Group Policy setting must be enabled. See [Troubleshooting](/docs/troubleshooting/agentos-service#agent-cant-press-ctrlaltdel) for details. ## Installation [#installation] **Download the installer** * [AMD64 (x86\_64)](https://files.askui.com/releases/Installer/Latest/AskUI-Agent-OS-Latest-Service-Installer-Win-AMD64.exe) * [ARM64](https://files.askui.com/releases/Installer/Latest/AskUI-Agent-OS-Latest-Service-Installer-Win-ARM64.exe) **Run the installer** Run the EXE and approve the administrator elevation request. **Configure the service (optional)** The installer includes a **Service Configuration** screen: * **Connection Scope:** Use **Public (0.0.0.0)** if AgentOS needs to be reachable from another machine (e.g. a remote CI runner). Keep **Private (127.0.0.1)** for local use only. * **Log Level:** `Info` for production, `Debug` for troubleshooting. * **Allow CTRL+ALT+DEL:** Enable to automate the Windows lock/login screen (requires Group Policy, see [Troubleshooting](/docs/troubleshooting/agentos-service#agent-cant-press-ctrlaltdel)). * **Allow control of administrative applications:** Enable only if you need to automate apps running as administrator. Click **Next**, then **Install**. **Reboot if prompted** If the installer asks to reboot, restart the machine so all changes take effect. AskUI Desktop and the CLI automatically detect the service and use it instead of standalone mode. For silent/unattended installation, see [Silent (exe)](/docs/agentos/installation/silent). ## Uninstalling [#uninstalling] Always uninstall via **Windows Add/Remove Programs**. Do not remove the service from Windows Services manually. # Silent (exe) (/docs/agentos/installation/silent) Install AgentOS as a Windows service unattended, for scripts, CI/CD pipelines, or automated provisioning. ## Download [#download] ## Quick Start [#quick-start] Run from an **elevated** command prompt (Run as administrator): ```bash "AskUI-Agent-OS--Service-Installer-Win-AMD64.exe" /qn ``` Replace `` with the actual version number (e.g. `26.2.1.3` ). With custom options (replace `` with the full path to where you downloaded the installer): ```bash "" /qn APPDIR="C:\ASKUI" /l*vx installer.log ``` Append `/l*vx installer.log` to any command to write detailed installer logs. ## Parameters [#parameters] | Parameter | Default | Description | | ----------------------------------------- | ------------------------------------------- | ------------------------------------------------------------------------ | | `APPDIR` | `C:\Program Files\askui GmbH\AskUI AgentOS` | Install location. Must be accessible to all users. | | `SERVICE_SUBNET` | `127.0.0.1` | Network subnet the service listens on. | | `SERVICE_LISTENING_PORT` | `26000` | Main service port. | | `SERVICE_LOG_LEVEL` | `Debug` | Log verbosity: `Info` or `Debug`. | | `SERVICE_ENABLE_SAS` | `1` | Enable Secure Attention Sequence (CTRL+ALT+DEL). `0` to disable. | | `SERVICE_EXECUTION_ENGINE_ELEVATED` | `0` | Run execution engine with elevated privileges. `1` to enable. | | `SERVICE_EXECUTION_ENGINE_LISTENING_PORT` | `23000` | Execution engine port. | | `SERVICE_EXECUTION_ENGINE_VISIBLE` | `0` | Show execution engine UI/process. `1` to make visible. | | `SERVICE_TRANSFER_SESSION_ON_DISCONNECT` | `1` | Transfer remote sessions on disconnect (RDP resilience). `0` to disable. | **Example with multiple parameters:** ```bash "" /qn SERVICE_LISTENING_PORT=27000 SERVICE_LOG_LEVEL=Info SERVICE_ENABLE_SAS=1 /l*vx installer.log ``` ## Runtime Configuration [#runtime-configuration] After installation, the service can be tuned by editing: ``` C:\Program Files\askui GmbH\AskUI AgentOS\Configs\AskuiCoreService.json ``` After changing this file, **restart the computer** for changes to take effect. # Standalone (/docs/agentos/installation/standalone) Install AgentOS locally in standalone mode, it runs as a regular process in your user session. This is the fastest way to get going on a development machine. AgentOS is proprietary software, **free for non-commercial use**. For commercial deployments a license is required, see [pricing](https://www.askui.com/enterprise) for details. ## Prerequisites [#prerequisites] * **Windows**: **macOS**, or **Linux**, see [System Requirements](/docs/agentos/reference/system-requirements) for supported versions. * **macOS only:** grant **Accessibility** and **Screen Recording** permissions to the app running AgentOS, under **System Settings → Privacy & Security**. ## Install [#install] AgentOS ships with the AskUI Desktop installer, so most users get it automatically when they [install AskUI Desktop](/docs/get-started/install-desktop). A dedicated AgentOS installer for machines without AskUI Desktop is in progress. For now, use the AskUI Desktop installer to get AgentOS on a machine. ## What's next? [#whats-next] * [Runtime Modes](/docs/agentos/understanding/runtime-modes), standalone vs OS service. * [Control Modes](/docs/agentos/understanding/control-modes), Host vs Companion. * [Deploy as a Service](/docs/agentos/installation/service), for CI and headless VMs.