> ## Documentation Index
> Fetch the complete documentation index at: https://docs.askui.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Types

<a id="askui.ActModel" />

## askui.ActModel

```python theme={null}
class ActModel(abc.ABC)
```

Abstract base class for models that can execute autonomous actions.

Models implementing this interface can be used with the `VisionAgent.act()`.

**Example**:

```python theme={null}
from askui import (
    ActModel,
    MessageParam,
    OnMessageCb,
    VisionAgent,
)
from typing_extensions import override

class MyActModel(ActModel):
    @override
    def act(
        self,
        messages: list[MessageParam],
        model_choice: str,
        on_message: OnMessageCb | None = None,
    ) -> None:
        pass  # implement action logic here

with VisionAgent(models={"my-act": MyActModel()}) as agent:
    agent.act("search for flights", model="my-act")
```

<a id="askui.exceptions.AiElementNotFound" />

## askui.exceptions.AiElementNotFound

```python theme={null}
class AiElementNotFound(ValueError)
```

Exception raised when an AI element is not found.

**Arguments**:

* `name` *str* - The name of the AI element that was not found.
* `locations` *list\[pathlib.Path]* - The locations that were searched for the AI element.

<a id="askui.exceptions.AskUiApiError" />

## askui.exceptions.AskUiApiError

```python theme={null}
class AskUiApiError(Exception)
```

Base exception for AskUI API errors.

This exception is raised when there is an error communicating with the AskUI API.
It serves as a base class for more specific API-related exceptions.

**Arguments**:

* `message` *str* - The error message.

<a id="askui.exceptions.AskUiApiRequestFailedError" />

## askui.exceptions.AskUiApiRequestFailedError

```python theme={null}
class AskUiApiRequestFailedError(AskUiApiError)
```

Exception raised when an API response is not as expected.

This exception is raised when the API returns a response that cannot be processed
or indicates an error condition. It includes the HTTP status code and error message
from the API response.

**Arguments**:

* `status_code` *int* - The HTTP status code from the API response.
* `message` *str* - The error message from the API response.

<a id="askui.exceptions.AutomationError" />

## askui.exceptions.AutomationError

```python theme={null}
class AutomationError(Exception)
```

Exception raised when the automation step cannot complete.

**Arguments**:

* `message` *str* - The error message.

<a id="askui.exceptions.ElementNotFoundError" />

## askui.exceptions.ElementNotFoundError

```python theme={null}
class ElementNotFoundError(AutomationError)
```

Exception raised when an element cannot be located.

**Arguments**:

* `message` *str* - The error message.

<a id="askui.exceptions.QueryNoResponseError" />

## askui.exceptions.QueryNoResponseError

```python theme={null}
class QueryNoResponseError(AutomationError)
```

Exception raised when a query does not return a response.

**Arguments**:

* `message` *str* - The error message.
* `query` *str* - The query that was made.

<a id="askui.exceptions.QueryUnexpectedResponseError" />

## askui.exceptions.QueryUnexpectedResponseError

```python theme={null}
class QueryUnexpectedResponseError(AutomationError)
```

Exception raised when a query returns an unexpected response.

**Arguments**:

* `message` *str* - The error message.
* `query` *str* - The query that was made.
* `response` *Any* - The response that was received.

<a id="askui.exceptions.ModelNotFoundError" />

## askui.exceptions.ModelNotFoundError

```python theme={null}
class ModelNotFoundError(AutomationError)
```

Exception raised when an invalid model is used.

**Arguments**:

* `model` *str | ModelComposition* - The model that was used.
* `model_type` *Literal\["Act", "Locator (locate)", "Query (get/extract)"]* - The type of model that was used.

<a id="askui.exceptions.ModelTypeMismatchError" />

## askui.exceptions.ModelTypeMismatchError

```python theme={null}
class ModelTypeMismatchError(ModelNotFoundError)
```

<a id="askui.GetModel" />

## askui.GetModel

```python theme={null}
class GetModel(abc.ABC)
```

Abstract base class for models that can extract information from images.

Models implementing this interface can be used with the `get()` method of
`VisionAgent`
to extract information from screenshots or other images. These models analyze visual
content and return structured or unstructured information based on queries.

**Example**:

```python theme={null}
from askui import GetModel, VisionAgent, ResponseSchema, ImageSource
from typing import Type

class MyGetModel(GetModel):
    def get(
        self,
        query: str,
        image: ImageSource,
        response_schema: Type[ResponseSchema] | None,
        model_choice: str,
    ) -> ResponseSchema | str:
        # Implement custom get logic
        return "Custom response"

with VisionAgent(models={"my-get": MyGetModel()}) as agent:
    result = agent.get("what's on screen?", model="my-get")
```

<a id="askui.ImageSource" />

## askui.ImageSource

```python theme={null}
class ImageSource(RootModel)
```

A Pydantic model that represents an image source and provides methods to convert it to different formats.

The model can be initialized with:

* A PIL Image object
* A file path (str or pathlib.Path)
* A data URL string

**Attributes**:

* `root` *PILImage.Image* - The underlying PIL Image object.

**Arguments**:

* `root` *Img* - The image source to load from.

<a id="askui.Img" />

## askui.Img

```python theme={null}
Img = Union[str, Path, PILImage.Image]
```

Type of the input images for `askui.VisionAgent.get()`, `askui.VisionAgent.locate()`, etc.

Accepts:

* `PIL.Image.Image`
* Relative or absolute file path (`str` or `pathlib.Path`)
* Data URL (e.g., `"data:image/png;base64,..."`)

<a id="askui.LocateModel" />

## askui.LocateModel

```python theme={null}
class LocateModel(abc.ABC)
```

Abstract base class for models that can locate UI elements in images.

Models implementing this interface can be used with the `click()`, `locate()`, and
`mouse_move()` methods of `VisionAgent` to find UI elements on screen. These models
analyze visual content to determine the coordinates of elements based on
descriptions or locators.

**Example**:

```python theme={null}
from askui import LocateModel, VisionAgent, Locator, ImageSource, Point
from askui.models import ModelComposition

class MyLocateModel(LocateModel):
    def locate(
        self,
        locator: str | Locator,
        image: ImageSource,
        model_choice: ModelComposition | str,
    ) -> Point:
        # Implement custom locate logic
        return (100, 100)

with VisionAgent(models={"my-locate": MyLocateModel()}) as agent:
    agent.click("button", model="my-locate")
```

<a id="askui.Model" />

## askui.Model

```python theme={null}
Model = ActModel | GetModel | LocateModel
```

Union type of all abstract model classes.

This type represents any model that can be used with `VisionAgent`, whether it's an
`ActModel`, `GetModel`, or `LocateModel`. It's useful for type hints when you need to
work with models in a generic way.

<a id="askui.ModelComposition" />

## askui.ModelComposition

```python theme={null}
class ModelComposition(RootModel[list[ModelDefinition]])
```

A composition of models (list of `ModelDefinition`) to be used for a task, e.g., locating an element on the screen to be able to click on it or extracting text from an image.

<a id="askui.ModelDefinition" />

## askui.ModelDefinition

```python theme={null}
class ModelDefinition(BaseModel)
```

A definition of a model.

**Arguments**:

* `task` *str* - The task the model is trained for, e.g., end-to-end OCR (`"e2e_ocr"`) or object detection (`"od"`)
* `architecture` *str* - The architecture of the model, e.g., `"easy_ocr"` or `"yolo"`
* `version` *str* - The version of the model
* `interface` *str* - The interface the model is trained for, e.g., `"online_learning"`
* `use_case` *str, optional* - The use case the model is trained for. In the case of workspace specific AskUI models, this is often the workspace id but with "-" replaced by "\_". Defaults to `"00000000_0000_0000_0000_000000000000"` (custom null value).
* `tags` *list\[str], optional* - Tags for identifying the model that cannot be represented by other properties, e.g., `["trained", "word_level"]`

<a id="askui.ModelRegistry" />

## askui.ModelRegistry

```python theme={null}
ModelRegistry = dict[str, Model | Callable[[], Model]]
```

Type definition for model registry.

A dictionary mapping model names to either model instances or factory functions (for
lazy initialization on first use) that create model instances. Used to register custom
models with `VisionAgent`.

**Example**:

```python theme={null}
from askui import ModelRegistry, ActModel

class MyModel(ActModel):
    def act(self, goal: str, model_choice: str) -> None:
        pass

# Registry with model instance
registry: ModelRegistry = {
    "my-model": MyModel()
}

# Registry with model factory
def create_model() -> ActModel:
    return MyModel()

registry_with_factory: ModelRegistry = {
    "my-model": create_model
}
```

<a id="askui.models.ModelName" />

## askui.models.ModelName

```python theme={null}
class ModelName(str, Enum)
```

Enumeration of all available model names in AskUI.

This enum provides type-safe access to model identifiers used throughout the
library. Each model name corresponds to a specific AI model or model composition
that can be used for different tasks like acting, getting information, or locating
elements.

<a id="askui.models.OpenRouterGetModel" />

## askui.models.OpenRouterGetModel

```python theme={null}
class OpenRouterGetModel(GetModel)
```

Get model for OpenRouter.

<a id="askui.models.OpenRouterSettings" />

## askui.models.OpenRouterSettings

```python theme={null}
class OpenRouterSettings(BaseModel)
```

Settings for OpenRouter.

<a id="askui.ModifierKey" />

## askui.ModifierKey

```python theme={null}
ModifierKey = Literal["command", "alt", "control", "shift", "right_shift"]
```

Modifier keys for keyboard actions.

<a id="askui.PcKey" />

## askui.PcKey

```python theme={null}
PcKey = Literal[
    "backspace",
    "delete",
    "enter",
    "tab",
    "escape",
    "up",
    "down",
    "right",
    "left",
    "home",
    "end",
    "pageup",
    "pagedown",
    "f1",
    "f2",
    "f3",
    "f4",
    "f5",
    "f6",
    "f7",
    "f8",
    "f9",
    "f10",
    "f11",
    "f12",
    "space",
    "0",
    "1",
    "2",
    "3",
    "4",
    "5",
    "6",
    "7",
    "8",
    "9",
    "a",
    "b",
    "c",
    "d",
    "e",
    "f",
    "g",
    "h",
    "i",
    "j",
    "k",
    "l",
    "m",
    "n",
    "o",
    "p",
    "q",
    "r",
    "s",
    "t",
    "u",
    "v",
    "w",
    "x",
    "y",
    "z",
    "A",
    "B",
    "C",
    "D",
    "E",
    "F",
    "G",
    "H",
    "I",
    "J",
    "K",
    "L",
    "M",
    "N",
    "O",
    "P",
    "Q",
    "R",
    "S",
    "T",
    "U",
    "V",
    "W",
    "X",
    "Y",
    "Z",
    "!",
    '"',
    "#",
    "$",
    "%",
    "&",
    "'",
    "(",
    ")",
    "*",
    "+",
    ",",
    "-",
    ".",
    "/",
    ":",
    ";",
    "<",
    "=",
    ">",
    "?",
    "@",
    "[",
    "\\",
    "]",
    "^",
    "_",
    "`",
    "{",
    "|",
    "}",
    "~",
]
```

PC keys for keyboard actions.

<a id="askui.models.Point" />

## askui.models.Point

```python theme={null}
Point = tuple[int, int]
```

A tuple of two integers representing the coordinates of a point on the screen.

<a id="askui.ResponseSchema" />

## askui.ResponseSchema

```python theme={null}
ResponseSchema = TypeVar(
    'ResponseSchema', ResponseSchemaBase, str, bool, int, float
)
```

Type of the responses of data extracted, e.g., using `askui.VisionAgent.get()`.

The following types are allowed:

* `ResponseSchemaBase`: Custom Pydantic models that extend `ResponseSchemaBase`
* `str`: String responses
* `bool`: Boolean responses
* `int`: Integer responses
* `float`: Floating point responses

Usually, serialized as a JSON schema, e.g., `str` as `{"type": "string"}`, to be passed to model(s).
Also used for validating the responses of the model(s) used for data extraction.

<a id="askui.ResponseSchemaBase" />

## askui.ResponseSchemaBase

```python theme={null}
class ResponseSchemaBase(BaseModel)
```

Base class for response schemas to be used for defining the response of data extraction, e.g., using `askui.VisionAgent.get()`.

This class extends Pydantic's BaseModel and adds constraints and configuration on top so that it can be used with models to define the schema (type) of the data to be extracted.

**Example**:

```python theme={null}
class UrlResponse(ResponseSchemaBase):
    url: str

# nested models should also extend ResponseSchemaBase
class NestedResponse(ResponseSchemaBase):
    nested: UrlResponse

# metadata, e.g., `examples` or `description` of `Field`, is generally also passed to and considered by the models
class UrlResponse(ResponseSchemaBase):
    url: str = Field(description="The URL of the response. Should used `"https"` scheme.", examples=["https://www.example.com"])
```