AskUIDocs
Best Practices

Agent behavior

How to shape the way the agent acts, what to write in each prompt file, error discipline, device facts, UI knowledge, the report contract, and per-folder rules.

The system prompt is assembled at run start from the four Markdown files in prompts/, plus each folder's rules.md. It applies to every test in the project. When tests fail consistently, this is often the fix, which file holds what: What goes where.

System capabilities: error discipline

prompts/system_capabilities.md defines the agent's identity and its always-on rules. The scaffolded default is a good starting point, edit with care; a small change here changes every test.

Its core is the error discipline:

prompts/system_capabilities.md (excerpt)
## Error Handling
- You have a maximum of **2 attempts per step**. If a step does not
  succeed after 2 attempts, stop immediately. Do not try a third time.
- Never try creative or alternative ways to accomplish a step that
  didn't work as written.
  • Two attempts, then FAILED: no third try, no workarounds, no navigating back to retry. A green run means the steps actually passed, not that the agent improvised its way to "done".
  • Infrastructure errors are different: connection lost, session expired, RPC error → status BROKEN, a diagnosis hint in the report, and exception_tool aborts the whole run (nothing can pass on broken infrastructure). The triage this enables: Analyzing failures.

Device information: only what the agent can't see

prompts/device_information.md holds quirks of this machine that the screen doesn't show. The agent sees the screen, OS and display basics rarely need describing.

prompts/device_information.md
* Always execute on display 1 — display 2 shows the corporate dashboard.
* The corporate lock screen appears after 5 minutes of inactivity.

Swap this file to retarget the project at a different machine without touching anything else.

UI information: your biggest lever

prompts/ui_information.md is where the agent learns your application. The agent has no built-in knowledge of your app, everything it must recognise has to be written down. This is the file you invest the most in, and the first place to look when the agent wanders.

prompts/ui_information.md
You are operating: Contoso Orders — order entry for the sales back office.

## Navigation
The application has these primary areas:
- Dashboard — overview & KPIs
- Records — data entry & search
- Settings — account & configuration

## State indicators
Login state is shown top-right: a green dot means signed in, red means
the session expired — sign in again.

## Dialogs & quirks
Destructive actions open a centered confirmation dialog that must be
confirmed or dismissed before continuing. The confirmation pop-up stays
open until you click OK — clicking elsewhere does not close it.

What belongs here:

  • Navigation: the primary areas and where key actions live.
  • State indicators: what "signed in", "loading", "error" look like, so the agent reads the screen correctly.
  • Non-standard interactions: anything that would surprise a first-time user: drag-and-drop patterns, lazy loading, single-page navigation.
  • Naming conventions: what areas and views are called, so tests can reference them by name.
  • What not to do: destructive actions that are easy to trigger.

Report format: the report contract

prompts/report_format.md defines the report the agent writes: the Markdown structure and the status vocabulary (PASSED, FAILED, SKIPPED, WARN, BROKEN). Keep the standard vocabulary, the runner parses the report's **Status:** line, and the Runs and Dashboard pages count results from it. Tailor the sections if your organisation needs different evidence; the shape is explained in Run report.

Rules: targeted patches per folder

A rules.md next to your tests patches the prompt for that folder only, use it instead of editing system_capabilities.md when one suite needs different behaviour. A good rule reads like a bug fix:

  • How the agent identifies the situation: what it sees on screen.
  • What to do: the exact sequence.
  • What not to do: rule out the wrong approach explicitly.
tests/rules.md
## Interaction
- Never use keyboard shortcuts — operate the application through its
  menus, like our users do.

## Recoveries
- If the scanner stops responding, run the restart_scanner tool and
  repeat the step. Do not mark the step FAILED on the first scanner
  timeout.

Style principles

  • Plain Markdown, one concern per ## section: that's what the scaffold ships and what keeps the files auditable.
  • One language: clear English throughout the prompt, tests, and all inputs. Mixed languages degrade performance.
  • No contradictions: a rule in rules.md that conflicts with system_capabilities.md produces undefined behaviour. Review across files.
  • Overly specific is right: if you feel you are being too detailed, you are doing it right. Only you know your UI.

After a prompt change, run the affected folder and read the run report, the Agent Interpretation per step shows whether the new wording landed the way you meant it.

On this page