> ## Documentation Index
> Fetch the complete documentation index at: https://docs.askui.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Tools

<a id="askui.tools.AgentOs" />

## askui.tools.AgentOs

```python theme={null}
class AgentOs(ABC)
```

Abstract base class for Agent OS. Cannot be instantiated directly.

This class defines the interface for operating system interactions including mouse control,
keyboard input, and screen capture functionality. Implementations should provide concrete
functionality for these abstract methods.

<a id="askui.tools.AgentOs.click" />

#### click

```python theme={null}
def click(
    button: Literal["left", "middle", "right"] = "left",
    count: int = 1
) -> None
```

Simulates clicking a mouse button.

**Arguments**:

* `button` *Literal\["left", "middle", "right"], optional* - The mouse button to click. Defaults to `"left"`.
* `count` *int, optional* - Number of times to click. Defaults to `1`.

<a id="askui.tools.AgentOs.connect" />

#### connect

```python theme={null}
def connect() -> None
```

Establishes a connection to the Agent OS.

This method is called before performing any OS-level operations.
It handles any necessary setup or initialization required for the OS interaction.

<a id="askui.tools.AgentOs.disconnect" />

#### disconnect

```python theme={null}
def disconnect() -> None
```

Terminates the connection to the Agent OS.

This method is called after all OS-level operations are complete.
It handles any necessary cleanup or resource release.

<a id="askui.tools.AgentOs.keyboard_pressed" />

#### keyboard\_pressed

```python theme={null}
def keyboard_pressed(
    key: PcKey | ModifierKey,
    modifier_keys: list[ModifierKey] | None = None
) -> None
```

Simulates pressing and holding a keyboard key.

**Arguments**:

* `key` *PcKey | ModifierKey* - The key to press.
* `modifier_keys` *list\[ModifierKey] | None, optional* - List of modifier keys to press along with the main key. Defaults to `None`.

<a id="askui.tools.AgentOs.keyboard_release" />

#### keyboard\_release

```python theme={null}
def keyboard_release(
    key: PcKey | ModifierKey,
    modifier_keys: list[ModifierKey] | None = None
) -> None
```

Simulates releasing a keyboard key.

**Arguments**:

* `key` *PcKey | ModifierKey* - The key to release.
* `modifier_keys` *list\[ModifierKey] | None, optional* - List of modifier keys to release along with the main key. Defaults to `None`.

<a id="askui.tools.agent_os.AgentOs.keyboard_tap" />

#### keyboard\_tap

```python theme={null}
def keyboard_tap(
    key: PcKey | ModifierKey,
    modifier_keys: list[ModifierKey] | None = None,
    count: int = 1
) -> None
```

Simulates pressing and immediately releasing a keyboard key.

**Arguments**:

* `key` *PcKey | ModifierKey* - The key to tap.
* `modifier_keys` *list\[ModifierKey] | None, optional* - List of modifier keys to
  press along with the main key. Defaults to `None`.
* `count` *int, optional* - The number of times to tap the key. Defaults to `1`.

<a id="askui.tools.agent_os.AgentOs.mouse_move" />

#### mouse\_move

```python theme={null}
def mouse_move(x: int, y: int) -> None
```

Moves the mouse cursor to specified screen coordinates.

**Arguments**:

* `x` *int* - The horizontal coordinate (in pixels) to move to.
* `y` *int* - The vertical coordinate (in pixels) to move to.

<a id="askui.tools.AgentOs.mouse_down" />

#### mouse\_down

```python theme={null}
def mouse_down(button: Literal["left", "middle", "right"] = "left") -> None
```

Simulates pressing and holding a mouse button.

**Arguments**:

* `button` *Literal\["left", "middle", "right"], optional* - The mouse button to press. Defaults to `"left"`.

<a id="askui.tools.AgentOs.mouse_scroll" />

#### mouse\_scroll

```python theme={null}
def mouse_scroll(x: int, y: int) -> None
```

Simulates scrolling the mouse wheel.

**Arguments**:

* `x` *int* - The horizontal scroll amount. Positive values scroll right, negative values scroll left.
* `y` *int* - The vertical scroll amount. Positive values scroll down, negative values scroll up.

<a id="askui.tools.AgentOs.mouse_up" />

#### mouse\_up

```python theme={null}
def mouse_up(button: Literal["left", "middle", "right"] = "left") -> None
```

Simulates releasing a mouse button.

**Arguments**:

* `button` *Literal\["left", "middle", "right"], optional* - The mouse button to release. Defaults to `"left"`.

<a id="askui.tools.agent_os.AgentOs.poll_event" />

#### poll\_event

```python theme={null}
def poll_event() -> InputEvent | None
```

Poll for a single input event.

IMPORTANT: This method is still experimental and may not work at all and may
change in the future.

<a id="askui.tools.agent_os.AgentOs.run_command" />

#### run\_command

```python theme={null}
def run_command(command: str, timeout_ms: int = 30000) -> None
```

Executes a shell command.

**Arguments**:

* `command` *str* - The command to execute.
* `timeout_ms` *int, optional* - The timeout for command
  execution in milliseconds. Defaults to `30000` (30 seconds).

<a id="askui.tools.AgentOs.screenshot" />

#### screenshot

```python theme={null}
def screenshot(report: bool = True) -> Image.Image
```

Captures a screenshot of the current display.

**Arguments**:

* `report` *bool, optional* - Whether to include the screenshot in reporting. Defaults to `True`.

**Returns**:

* `Image.Image` - A PIL Image object containing the screenshot.

<a id="askui.tools.AgentOs.set_display" />

#### set\_display

```python theme={null}
def set_display(display: int = 1) -> None
```

Sets the active display for screen interactions.

**Arguments**:

* `display` *int, optional* - The display ID to set as active.

<a id="askui.tools.agent_os.AgentOs.start_listening" />

#### start\_listening

```python theme={null}
def start_listening() -> None
```

Start listening for mouse and keyboard events.

IMPORTANT: This method is still experimental and may not work at all and may
change in the future.

<a id="askui.tools.agent_os.AgentOs.stop_listening" />

#### stop\_listening

```python theme={null}
def stop_listening() -> None
```

Stop listening for mouse and keyboard events.

IMPORTANT: This method is still experimental and may not work at all and may
change in the future.

<a id="askui.tools.AgentOs.type" />

#### type

```python theme={null}
def type(text: str, typing_speed: int = 50) -> None
```

Simulates typing text as if entered on a keyboard.

**Arguments**:

* `text` *str* - The text to be typed.
* `typing_speed` *int, optional* - The speed of typing in characters per minute. Defaults to `50`.

<a id="askui.tools.askui.AskUiControllerClient" />

## askui.tools.askui.AskUiControllerClient

```python theme={null}
class AskUiControllerClient(AgentOs)
```

Implementation of `AgentOs` that communicates with the AskUI Remote Device Controller via gRPC.

**Arguments**:

* `reporter` *Reporter* - Reporter used for reporting with the `"AgentOs"`.
* `display` *int, optional* - Display number to use. Defaults to `1`.
* `controller_server` *AskUiControllerServer | None, optional* - Custom controller server. Defaults to `ControllerServer`.

<a id="askui.tools.askui.AskUiControllerClient.__enter__" />

#### \_\_enter\_\_

```python theme={null}
def __enter__() -> Self
```

Context manager entry point that establishes the connection.

**Returns**:

* `Self` - The instance of AskUiControllerClient.

<a id="askui.tools.askui.AskUiControllerClient.__exit__" />

#### \_\_exit\_\_

```python theme={null}
def __exit__(exc_type, exc_value, traceback) -> None
```

Context manager exit point that disconnects the client.

**Arguments**:

* `exc_type` - The exception type if an exception was raised.
* `exc_value` - The exception value if an exception was raised.
* `traceback` - The traceback if an exception was raised.

<a id="askui.tools.askui.AskUiControllerClient.click" />

#### click

```python theme={null}
def click(
    button: Literal['left', 'middle', 'right'] = 'left',
    count: int = 1
) -> None
```

Simulates clicking a mouse button.

**Arguments**:

* `button` *Literal\["left", "middle", "right"], optional* - The mouse button to click. Defaults to `"left"`.
* `count` *int, optional* - Number of times to click. Defaults to `1`.

<a id="askui.tools.askui.AskUiControllerClient.connect" />

#### connect

```python theme={null}
def connect() -> None
```

Establishes a connection to the AskUI Remote Device Controller.

This method starts the controller server, establishes a gRPC channel,
creates a session, and sets up the initial display.

<a id="askui.tools.askui.AskUiControllerClient.disconnect" />

#### disconnect

```python theme={null}
def disconnect() -> None
```

Terminates the connection to the AskUI Remote Device Controller.

This method stops the execution, ends the session, closes the gRPC channel,
and stops the controller server.

<a id="askui.tools.askui.AskUiControllerClient.keyboard_pressed" />

#### keyboard\_pressed

```python theme={null}
def keyboard_pressed(
    key: PcKey | ModifierKey,
    modifier_keys: list[ModifierKey] | None = None
) -> None
```

Simulates pressing and holding a keyboard key.

**Arguments**:

* `key` *PcKey | ModifierKey* - The key to press.
* `modifier_keys` *list\[ModifierKey] | None, optional* - List of modifier keys to press along with the main key. Defaults to `None`.

<a id="askui.tools.askui.AskUiControllerClient.keyboard_release" />

#### keyboard\_release

```python theme={null}
def keyboard_release(
    key: PcKey | ModifierKey,
    modifier_keys: list[ModifierKey] | None = None
) -> None
```

Simulates releasing a keyboard key.

**Arguments**:

* `key` *PcKey | ModifierKey* - The key to release.
* `modifier_keys` *list\[ModifierKey] | None, optional* - List of modifier keys to release along with the main key. Defaults to `None`.

<a id="askui.tools.askui.askui_controller.AskUiControllerClient.keyboard_tap" />

#### keyboard\_tap

```python theme={null}
def keyboard_tap(
    key: PcKey | ModifierKey,
    modifier_keys: list[ModifierKey] | None = None,
    count: int = 1
) -> None
```

Press and immediately release a keyboard key.

**Arguments**:

* `key` *PcKey | ModifierKey* - The key to tap.
* `modifier_keys` *list\[ModifierKey] | None, optional* - List of modifier keys to
  press along with the main key. Defaults to `None`.
* `count` *int, optional* - The number of times to tap the key. Defaults to `1`.

<a id="askui.tools.askui.askui_controller.AskUiControllerClient.mouse_move" />

#### mouse\_move

```python theme={null}
def mouse_move(x: int, y: int) -> None
```

Moves the mouse cursor to specified screen coordinates.

**Arguments**:

* `x` *int* - The horizontal coordinate (in pixels) to move to.
* `y` *int* - The vertical coordinate (in pixels) to move to.

<a id="askui.tools.askui.AskUiControllerClient.mouse_down" />

#### mouse\_down

```python theme={null}
def mouse_down(button: Literal['left', 'middle', 'right'] = 'left') -> None
```

Simulates pressing and holding a mouse button.

**Arguments**:

* `button` *Literal\["left", "middle", "right"], optional* - The mouse button to press. Defaults to `"left"`.

<a id="askui.tools.askui.AskUiControllerClient.mouse_scroll" />

#### mouse\_scroll

```python theme={null}
def mouse_scroll(x: int, y: int) -> None
```

Simulates scrolling the mouse wheel.

**Arguments**:

* `x` *int* - The horizontal scroll amount. Positive values scroll right, negative values scroll left.
* `y` *int* - The vertical scroll amount. Positive values scroll down, negative values scroll up.

<a id="askui.tools.askui.AskUiControllerClient.mouse_up" />

#### mouse\_up

```python theme={null}
def mouse_up(button: Literal['left', 'middle', 'right'] = 'left') -> None
```

Simulates releasing a mouse button.

**Arguments**:

* `button` *Literal\["left", "middle", "right"], optional* - The mouse button to release. Defaults to `"left"`.

<a id="askui.tools.askui.AskUiControllerClient.screenshot" />

#### screenshot

```python theme={null}
def screenshot(report: bool = True) -> Image.Image
```

Captures a screenshot of the current display.

**Arguments**:

* `report` *bool, optional* - Whether to include the screenshot in reporting. Defaults to `True`.

**Returns**:

* `Image.Image` - A PIL Image object containing the screenshot.

<a id="askui.tools.askui.AskUiControllerClient.set_display" />

#### set\_display

```python theme={null}
def set_display(display: int = 1) -> None
```

Sets the active display for screen interactions.

**Arguments**:

* `display` *int, optional* - The display ID to set as active.

<a id="askui.tools.askui.AskUiControllerClient.type" />

#### type

```python theme={null}
def type(text: str, typing_speed: int = 50) -> None
```

Simulates typing text as if entered on a keyboard.

**Arguments**:

* `text` *str* - The text to be typed.
* `typing_speed` *int, optional* - The speed of typing in characters per second. Defaults to `50`.

<a id="askui.tools.askui.exceptions.AskUiControllerError" />

## askui.tools.askui.exceptions.AskUiControllerError

```python theme={null}
class AskUiControllerError(Exception)
```

Base exception for AskUI controller errors.

This exception is raised when there is an error in the AskUI controller (client),
which handles the communication with the AskUI controller (server).

**Arguments**:

* `message` *str* - The error message.

<a id="askui.tools.askui.exceptions.AskUiControllerOperationFailedError" />

## askui.tools.askui.exceptions.AskUiControllerOperationFailedError

```python theme={null}
class AskUiControllerOperationFailedError(AskUiControllerError)
```

Exception raised when a controller operation fails.

This exception is raised when a specific operation in the AskUI controller
fails, such as starting/stopping the controller or executing an action.

**Arguments**:

* `operation` *str* - The operation that failed.
* `error` *Exception* - The original error that caused the failure.

<a id="askui.tools.askui.exceptions.AskUiControllerOperationTimeoutError" />

## askui.tools.askui.exceptions.AskUiControllerOperationTimeoutError

```python theme={null}
class AskUiControllerOperationTimeoutError(AskUiControllerError)
```

Exception raised when a controller action times out.

This exception is raised when an action in the AskUI controller takes longer
than the expected timeout period to complete.

**Arguments**:

* `message` *str* - The error message.
* `timeout_seconds` *float | None* - Optional timeout duration in seconds.

<a id="askui.tools.askui.AskUiControllerServer" />

## askui.tools.askui.AskUiControllerServer

```python theme={null}
class AskUiControllerServer()
```

Concrete implementation of `ControllerServer` for managing the AskUI Remote Device Controller process.
Handles process discovery, startup, and shutdown for the native controller binary.

<a id="askui.tools.askui.AskUiControllerServer.start" />

#### start

```python theme={null}
def start(clean_up: bool = False) -> None
```

Start the controller process.

**Arguments**:

* `clean_up` *bool, optional* - Whether to clean up existing processes (only on Windows) before starting. Defaults to `False`.

<a id="askui.tools.askui.AskUiControllerServer.stop" />

#### stop

```python theme={null}
def stop(force: bool = False) -> None
```

Stop the controller process.

**Arguments**:

* `force` *bool, optional* - Whether to forcefully terminate the process. Defaults to `False`.

<a id="askui.tools.exceptions.ToolError" />

## askui.tools.exceptions.ToolError

```python theme={null}
class ToolError(Exception)
```

Raised when a tool encounters an error.

**Arguments**:

* `message` *str* - The error message.
* `result` *ToolResult* - The ToolResult that caused the error.

<a id="askui.tools.exceptions.ToolResult" />

## askui.tools.exceptions.ToolResult

```python theme={null}
class ToolResult()
```

Represents the result of a tool execution.

**Arguments**:

* `output` *str | None, optional* - The output of the tool.
* `error` *str | None, optional* - The error message of the tool.
* `base64_image` *str | None, optional* - The base64 image of the tool.
* `system` *str | None, optional* - The system message of the tool.