Types

​

askui.exceptions.AiElementNotFound

class AiElementNotFound(ValueError)

Exception raised when an AI element is not found.

Arguments:

• name str - The name of the AI element that was not found.
• locations list[pathlib.Path] - The locations that were searched for the AI element.

​

askui.exceptions.AskUiApiError

class AskUiApiError(Exception)

Base exception for AskUI API errors.

This exception is raised when there is an error communicating with the AskUI API. It serves as a base class for more specific API-related exceptions.

Arguments:

• message str - The error message.

​

askui.exceptions.AskUiApiRequestFailedError

class AskUiApiRequestFailedError(AskUiApiError)

Exception raised when an API response is not as expected.

This exception is raised when the API returns a response that cannot be processed or indicates an error condition. It includes the HTTP status code and error message from the API response.

Arguments:

• status_code int - The HTTP status code from the API response.
• message str - The error message from the API response.

​

askui.exceptions.AutomationError

class AutomationError(Exception)

Exception raised when the automation step cannot complete.

Arguments:

• message str - The error message.

​

askui.exceptions.ElementNotFoundError

class ElementNotFoundError(AutomationError)

Exception raised when an element cannot be located.

Arguments:

• message str - The error message.

​

askui.exceptions.QueryNoResponseError

class QueryNoResponseError(AutomationError)

Exception raised when a query does not return a response.

Arguments:

• message str - The error message.
• query str - The query that was made.

​

askui.exceptions.QueryUnexpectedResponseError

class QueryUnexpectedResponseError(AutomationError)

Exception raised when a query returns an unexpected response.

Arguments:

• message str - The error message.
• query str - The query that was made.
• response Any - The response that was received.

​

askui.exceptions.ModelNotFoundError

class ModelNotFoundError(AutomationError)

Exception raised when an invalid model is used.

Arguments:

• model str | ModelComposition - The model that was used.
• model_type Literal[“Act”, “Grounding (locate)”, “Query (get/extract)”] - The type of model that was used.

​

askui.Img

Img = Union[str, Path, PILImage.Image]

Type of the input images for askui.VisionAgent.get(), askui.VisionAgent.locate(), etc.

Accepts:

• PIL.Image.Image
• Relative or absolute file path (str or pathlib.Path)
• Data URL (e.g., "data:image/png;base64,...")

​

askui.ModelComposition

class ModelComposition(RootModel[list[ModelDefinition]])

A composition of models (list of ModelDefinition) to be used for a task, e.g., locating an element on the screen to be able to click on it or extracting text from an image.

​

askui.ModelDefinition

class ModelDefinition(BaseModel)

A definition of a model.

Arguments:

• task str - The task the model is trained for, e.g., end-to-end OCR ("e2e_ocr") or object detection ("od")
• architecture str - The architecture of the model, e.g., "easy_ocr" or "yolo"
• version str - The version of the model
• interface str - The interface the model is trained for, e.g., "online_learning"
• use_case str, optional - The use case the model is trained for. In the case of workspace specific AskUI models, this is often the workspace id but with ”-” replaced by ”_”. Defaults to "00000000_0000_0000_0000_000000000000" (custom null value).
• tags list[str], optional - Tags for identifying the model that cannot be represented by other properties, e.g., ["trained", "word_level"]

​

askui.ModifierKey

ModifierKey = Literal["command", "alt", "control", "shift", "right_shift"]

Modifier keys for keyboard actions.

​

askui.PcKey

PcKey = Literal[
    "backspace",
    "delete",
    "enter",
    "tab",
    "escape",
    "up",
    "down",
    "right",
    "left",
    "home",
    "end",
    "pageup",
    "pagedown",
    "f1",
    "f2",
    "f3",
    "f4",
    "f5",
    "f6",
    "f7",
    "f8",
    "f9",
    "f10",
    "f11",
    "f12",
    "space",
    "0",
    "1",
    "2",
    "3",
    "4",
    "5",
    "6",
    "7",
    "8",
    "9",
    "a",
    "b",
    "c",
    "d",
    "e",
    "f",
    "g",
    "h",
    "i",
    "j",
    "k",
    "l",
    "m",
    "n",
    "o",
    "p",
    "q",
    "r",
    "s",
    "t",
    "u",
    "v",
    "w",
    "x",
    "y",
    "z",
    "A",
    "B",
    "C",
    "D",
    "E",
    "F",
    "G",
    "H",
    "I",
    "J",
    "K",
    "L",
    "M",
    "N",
    "O",
    "P",
    "Q",
    "R",
    "S",
    "T",
    "U",
    "V",
    "W",
    "X",
    "Y",
    "Z",
    "!",
    '"',
    "#",
    "$",
    "%",
    "&",
    "'",
    "(",
    ")",
    "*",
    "+",
    ",",
    "-",
    ".",
    "/",
    ":",
    ";",
    "<",
    "=",
    ">",
    "?",
    "@",
    "[",
    "\\",
    "]",
    "^",
    "_",
    "`",
    "{",
    "|",
    "}",
    "~",
]

PC keys for keyboard actions.

​

askui.Point

Point = tuple[int, int]

A tuple of two integers representing the coordinates of a point on the screen.

​

askui.ResponseSchema

ResponseSchema = TypeVar(
    'ResponseSchema', ResponseSchemaBase, str, bool, int, float
)

Type of the responses of data extracted, e.g., using askui.VisionAgent.get().

The following types are allowed:

• ResponseSchemaBase: Custom Pydantic models that extend ResponseSchemaBase
• str: String responses
• bool: Boolean responses
• int: Integer responses
• float: Floating point responses

Usually, serialized as a JSON schema, e.g., str as {"type": "string"}, to be passed to model(s). Also used for validating the responses of the model(s) used for data extraction.

​

askui.ResponseSchemaBase

class ResponseSchemaBase(BaseModel)

Base class for response schemas to be used for defining the response of data extraction, e.g., using askui.VisionAgent.get().

This class extends Pydantic’s BaseModel and adds constraints and configuration on top so that it can be used with models to define the schema (type) of the data to be extracted.

Example:

class UrlResponse(ResponseSchemaBase):
    url: str

# nested models should also extend ResponseSchemaBase
class NestedResponse(ResponseSchemaBase):
    nested: UrlResponse

# metadata, e.g., `examples` or `description` of `Field`, is generally also passed to and considered by the models
class UrlResponse(ResponseSchemaBase):
    url: str = Field(description="The URL of the response. Should used `"https"` scheme.", examples=["https://www.example.com"])