This page provides a glossary of key AskUI terms and definitions to help you understand the core concepts and components of the platform.

Glossary

Starter Kit

Build AI agents that control desktop and mobile devices

Welcome to AskUI

Learn AskUI by building your first AI agent

Introduction

Install AskUI and set up your development environment

Installation

Build your first automation agent with AskUI

Your First Agent

Practical solutions for common AskUI tasks

Overview

Learn the concepts behind AI-powered UI automation

Frequently Asked Questions

Agent

Locators

Reporting

Retry

Tools

Types

Hardware and software requirements for running AskUI

System Requirements

Network configuration and connectivity requirements for AskUI

Network Requirements

Reference guide for available AI models and their specifications

AI Models

Status Page

Trust Center

TypeScript Docs

Community

Roadmap

Reference

Login

Support

Sign up

Explore real-world automation examples showcasing AskUI's capabilities

Example Automations Overview

Automating complex UI interactions in an online casino application

Online Casino Automation

Automating industrial control systems with drag interactions

PLC Water Tank Application

Building a game-playing AI agent with complex decision logic

Tic Tac Toe Agent

Automating mobile applications on Android devices

Android Automation

Automating automotive infotainment systems and embedded UIs

Infotainment System Testing

Automating e-commerce decisions and purchases on Amazon

Web Shopping Assistant

Learn how to build AI agents with AskUI's core capabilities

Use the act() command to accomplish tasks with natural language

Act towards goal

How to select and interact with UI elements using AskUI

Select UI Elements

Learn how to interact with UI elements using AskUI's powerful tools and actions

Interact with Elements

Step-by-step guide to configure and use different AI models with AskUI

Configure Different Models

Workspace Management

Learn how to manage members and their permissions in your AskUI workspace.

Members Management

Learn how to track your AskUI usage and manage billing information in the dashboard.

Usage & Billing Dashboard

Access Tokens

Change your subscription and manage invoices

Billing and Invoices

This page explains how to set up **AskUI with GitHub Actions**, including dependency setup, environment variables, test execution, and optional report generation.

Github Actions

This page explains how to set up and run **AskUI workflows in a CI/CD pipeline**, including dependencies, environment variables, and optional report generation, with a full example on **GitLab**.

Gitlab CI/CD

This page explains how to set up and run **AskUI workflows in Azure DevOps**, including pipeline configuration, environment variables, Docker setup, and report generation, with a full example on **GitHub**.

Azure DevOps

This page covers **AskUI Reporters**, a package for tracking automation steps. It explains **installation, available reporters, combining multiple reporters**, and how to create custom reporters using the AskUI **Reporter interface**.

Reporters

Welcome to the home of your new documentation

SSO with AskUI

Please use the links provided below for more information.

Troubleshooting

Solutions for common installation and setup problems

Installation and Setup Issues

Solutions for proxy, firewall, and network-related problems

Network and Connectivity Issues

Solutions for element selection, text recognition, and detection problems

Element Detection and OCR Issues

Solutions for problems with agent.get() and data extraction

Data Extraction Issues

Solutions for runtime errors and environment configuration issues

Runtime and Environment Errors

How to report issues, generate error reports, and get support

Reporting Bugs and Getting Help

Active issues and limitations in AskUI products

Current Known Issues

AskUI capabilities, features, and comparison with other AI automation solutions

Platform Overview

Understand the fundamental building blocks of AskUI

Core Components

Understand how AskUI processes and executes UI automation tasks

Workflow

Understanding AskUI's multi-layered AI system and how different models work together

AI Model Architecture

Best practices for using goal-based automation with AskUI

Agentic Mode

Best practices for finding and selecting UI elements with AskUI

Element Selection

Best practices for interacting with UI elements in AskUI

Interactions

Learn how to monitor and debug your AskUI automation workflows

Reporting and Logging

Learn about re-teaching vs training and how to improve model performance for your specific needs.

Improve Model Performance

Create Access Token

Delete Access Token

List Access Tokens

Converts a sensitive global-level access token into its stable identifier (access_token_id). The access token can be base64 encoded (similar to Authorization header) or passed raw. This enables clients to reference tokens without transmitting sensitive values, e.g., when trying to delete the token or retrieve usage for the token.

Lookup Global Access Token ID

Converts a sensitive workspace-level access token into its stable identifier (access_token_id) within the specified workspace. The access token can be base64 encoded (similar to Authorization header) or passed raw. This enables clients to reference tokens without transmitting sensitive values, e.g., when trying to delete the token or retrieve usage for the token.

Lookup Workspace Access Token ID

Create Agent Execution

List all agent executions matching the query parameters sorted anti-chronologically by when they were last updated (`updated_at`).

List Agent Executions

Update Agent Execution

Create Agent

List all agents matching the query parameters sorted anti-chronologically by when they were last updated (`updated_at`).

List Agents

Update Agent

Create Customer Portal Session

Retrieves the subscription details for the workspace, including Stripe product information.

Get workspace subscription details

Delete a file at the specified path.

- Deletes only one file with the given file path. Not bulk deletion.
- If there is no file at the specified path, the operation will be a no-op, i.e., it will still return a 204 status code.

Delete File

List files.

- To list files within a workspace, the `prefix` (query parameter) must start with `workspaces/{workspace_id}/`
- Cannot list files across multiple workspaces, i.e., if the `prefix` does not start with `workspaces/{workspace_id}/`, it will return an empty list.

List Files

Set http-only, secure (signed) cookies for accessing (only READ access) files via AWS CloudFront across all accessible workspaces. If no workspace id is provided, no cookies will be set.

Set Signed Cookies

Upload a file (max. 5 GB) to the specified path.

- Specify the `Content-Type` header for accurate file type handling.
- If `Content-Type` is omitted, the system will attempt to infer it.
- For workspace-specific uploads, use path: `workspaces/{workspace_id}/...`
- If a file with the same file path already exists, it will be overwritten.
- If there are unsupported characters in the file path, they will be removed or replaced.
- If the file path is longer than `1024` characters, the file path will be shortened starting from the end.
- Set `strict=true` to fail the request if the file path would be modified during sanitization (e.g., if the file path contains unsupported characters or is too long).

Term	Meaning
access token	Gives you access to the AskUI inference in combination with your workspace ID. Every access token has an expiry date and is assigned to exactly one workspace. See Managing Access Tokens.
action	A method in the AskUI Control Client API that describes an action to be taken against the operating system, e.g., `click()`, `type()`. See Vision Agent API Actions.
annotation	Marked area around an element with metadata including name, text, and bounding box coordinates.
automation	A system of multiple connected workflows.
bounding box	A rectangle described by coordinates that define an element’s location, visually displayed as a red rectangle.
credentials	A combination of workspace ID and access token, both tied to the same workspace.
element	A graphical user interface component (or ui element), e.g., a button, text, menu, text field etc.
inference	The process of annotating a user interface.
inference server	The backend system that performs inference (UI analysis and annotation).
instruction	A single AskUI directive, usually consisting of action + (optional) locator.
interactive annotation	Exploring the annotations of a user interface through an annotated screenshot.
locator	A description of an UI element that is used to (re-)locate (find) the element on the screen, e.g., when trying to click on it; can be a simple textual description like `"login button"` or more complex, potentially multi-modal, e.g., `loc.Image("path/to/image.png").below_of(loc.Text("Login"))`. See Vision Agent API Locators and Element Selection Guide.
UI Controller (legacy)	A service that controls inputs and observes visuals on the operating system.
AskUI Controller	The updated service that controls inputs and observes visuals on the operating system.
UI Control Client	Retrieves annotations from the inference server and executes input instructions on the operating system via the AskUI Controller.
AskUI Studio (former User Portal)	A platform where you can create access tokens for your workspace.
AskUI-Installer	The installer that sets up AskUI components on a machine.
AskUI Development Environment (ADE)	A terminal environment for setting up and managing AskUI components and projects.
AskUI Runner	A self-hosted component that downloads and runs workflows from AskUI Studio.
workflow	A set of instructions designed to complete a single task.
workspace	Every registered user has exactly one workspace, which can contain multiple access tokens.
workspace ID	The unique identifier for your workspace, used in combination with an access token to access AskUI inference.

Documentation

Tutorial

How-to Guides

Understanding AskUI

Glossary