askui.tools.AgentOs
click
button
Literal[“left”, “middle”, “right”], optional - The mouse button to click. Defaults to"left"
.count
int, optional - Number of times to click. Defaults to1
.
connect
disconnect
keyboard_pressed
key
PcKey | ModifierKey - The key to press.modifier_keys
list[ModifierKey] | None, optional - List of modifier keys to press along with the main key. Defaults toNone
.
keyboard_release
key
PcKey | ModifierKey - The key to release.modifier_keys
list[ModifierKey] | None, optional - List of modifier keys to release along with the main key. Defaults toNone
.
keyboard_tap
key
PcKey | ModifierKey - The key to tap.modifier_keys
list[ModifierKey] | None, optional - List of modifier keys to press along with the main key. Defaults toNone
.count
int, optional - The number of times to tap the key. Defaults to1
.
mouse_move
x
int - The horizontal coordinate (in pixels) to move to.y
int - The vertical coordinate (in pixels) to move to.
mouse_down
button
Literal[“left”, “middle”, “right”], optional - The mouse button to press. Defaults to"left"
.
mouse_scroll
x
int - The horizontal scroll amount. Positive values scroll right, negative values scroll left.y
int - The vertical scroll amount. Positive values scroll down, negative values scroll up.
mouse_up
button
Literal[“left”, “middle”, “right”], optional - The mouse button to release. Defaults to"left"
.
poll_event
run_command
command
str - The command to execute.timeout_ms
int, optional - The timeout for command execution in milliseconds. Defaults to30000
(30 seconds).
screenshot
report
bool, optional - Whether to include the screenshot in reporting. Defaults toTrue
.
Image.Image
- A PIL Image object containing the screenshot.
set_display
display
int, optional - The display ID to set as active.
start_listening
stop_listening
type
text
str - The text to be typed.typing_speed
int, optional - The speed of typing in characters per minute. Defaults to50
.
askui.tools.askui.AskUiControllerClient
AgentOs
that communicates with the AskUI Remote Device Controller via gRPC.
Arguments:
reporter
Reporter - Reporter used for reporting with the"AgentOs"
.display
int, optional - Display number to use. Defaults to1
.controller_server
AskUiControllerServer | None, optional - Custom controller server. Defaults toControllerServer
.
__enter__
Self
- The instance of AskUiControllerClient.
__exit__
exc_type
- The exception type if an exception was raised.exc_value
- The exception value if an exception was raised.traceback
- The traceback if an exception was raised.
click
button
Literal[“left”, “middle”, “right”], optional - The mouse button to click. Defaults to"left"
.count
int, optional - Number of times to click. Defaults to1
.
connect
disconnect
keyboard_pressed
key
PcKey | ModifierKey - The key to press.modifier_keys
list[ModifierKey] | None, optional - List of modifier keys to press along with the main key. Defaults toNone
.
keyboard_release
key
PcKey | ModifierKey - The key to release.modifier_keys
list[ModifierKey] | None, optional - List of modifier keys to release along with the main key. Defaults toNone
.
keyboard_tap
key
PcKey | ModifierKey - The key to tap.modifier_keys
list[ModifierKey] | None, optional - List of modifier keys to press along with the main key. Defaults toNone
.count
int, optional - The number of times to tap the key. Defaults to1
.
mouse_move
x
int - The horizontal coordinate (in pixels) to move to.y
int - The vertical coordinate (in pixels) to move to.
mouse_down
button
Literal[“left”, “middle”, “right”], optional - The mouse button to press. Defaults to"left"
.
mouse_scroll
x
int - The horizontal scroll amount. Positive values scroll right, negative values scroll left.y
int - The vertical scroll amount. Positive values scroll down, negative values scroll up.
mouse_up
button
Literal[“left”, “middle”, “right”], optional - The mouse button to release. Defaults to"left"
.
screenshot
report
bool, optional - Whether to include the screenshot in reporting. Defaults toTrue
.
Image.Image
- A PIL Image object containing the screenshot.
set_display
display
int, optional - The display ID to set as active.
type
text
str - The text to be typed.typing_speed
int, optional - The speed of typing in characters per second. Defaults to50
.
askui.tools.askui.exceptions.AskUiControllerError
message
str - The error message.
askui.tools.askui.exceptions.AskUiControllerOperationFailedError
operation
str - The operation that failed.error
Exception - The original error that caused the failure.
askui.tools.askui.exceptions.AskUiControllerOperationTimeoutError
message
str - The error message.timeout_seconds
float | None - Optional timeout duration in seconds.
askui.tools.askui.AskUiControllerServer
ControllerServer
for managing the AskUI Remote Device Controller process.
Handles process discovery, startup, and shutdown for the native controller binary.
start
clean_up
bool, optional - Whether to clean up existing processes (only on Windows) before starting. Defaults toFalse
.
stop
force
bool, optional - Whether to forcefully terminate the process. Defaults toFalse
.
askui.tools.exceptions.ToolError
message
str - The error message.result
ToolResult - The ToolResult that caused the error.
askui.tools.exceptions.ToolResult
output
str | None, optional - The output of the tool.error
str | None, optional - The error message of the tool.base64_image
str | None, optional - The base64 image of the tool.system
str | None, optional - The system message of the tool.