button
Literal[“left”, “middle”, “right”], optional - The mouse button to click. Defaults to "left"
.count
int, optional - Number of times to click. Defaults to 1
.key
PcKey | ModifierKey - The key to press.modifier_keys
list[ModifierKey] | None, optional - List of modifier keys to press along with the main key. Defaults to None
.key
PcKey | ModifierKey - The key to release.modifier_keys
list[ModifierKey] | None, optional - List of modifier keys to release along with the main key. Defaults to None
.key
PcKey | ModifierKey - The key to tap.modifier_keys
list[ModifierKey] | None, optional - List of modifier keys to
press along with the main key. Defaults to None
.count
int, optional - The number of times to tap the key. Defaults to 1
.x
int - The horizontal coordinate (in pixels) to move to.y
int - The vertical coordinate (in pixels) to move to.button
Literal[“left”, “middle”, “right”], optional - The mouse button to press. Defaults to "left"
.x
int - The horizontal scroll amount. Positive values scroll right, negative values scroll left.y
int - The vertical scroll amount. Positive values scroll down, negative values scroll up.button
Literal[“left”, “middle”, “right”], optional - The mouse button to release. Defaults to "left"
.command
str - The command to execute.timeout_ms
int, optional - The timeout for command
execution in milliseconds. Defaults to 30000
(30 seconds).report
bool, optional - Whether to include the screenshot in reporting. Defaults to True
.Image.Image
- A PIL Image object containing the screenshot.display
int, optional - The display ID to set as active.text
str - The text to be typed.typing_speed
int, optional - The speed of typing in characters per minute. Defaults to 50
.AgentOs
that communicates with the AskUI Remote Device Controller via gRPC.
Arguments:
reporter
Reporter - Reporter used for reporting with the "AgentOs"
.display
int, optional - Display number to use. Defaults to 1
.controller_server
AskUiControllerServer | None, optional - Custom controller server. Defaults to ControllerServer
.Self
- The instance of AskUiControllerClient.exc_type
- The exception type if an exception was raised.exc_value
- The exception value if an exception was raised.traceback
- The traceback if an exception was raised.button
Literal[“left”, “middle”, “right”], optional - The mouse button to click. Defaults to "left"
.count
int, optional - Number of times to click. Defaults to 1
.key
PcKey | ModifierKey - The key to press.modifier_keys
list[ModifierKey] | None, optional - List of modifier keys to press along with the main key. Defaults to None
.key
PcKey | ModifierKey - The key to release.modifier_keys
list[ModifierKey] | None, optional - List of modifier keys to release along with the main key. Defaults to None
.key
PcKey | ModifierKey - The key to tap.modifier_keys
list[ModifierKey] | None, optional - List of modifier keys to
press along with the main key. Defaults to None
.count
int, optional - The number of times to tap the key. Defaults to 1
.x
int - The horizontal coordinate (in pixels) to move to.y
int - The vertical coordinate (in pixels) to move to.button
Literal[“left”, “middle”, “right”], optional - The mouse button to press. Defaults to "left"
.x
int - The horizontal scroll amount. Positive values scroll right, negative values scroll left.y
int - The vertical scroll amount. Positive values scroll down, negative values scroll up.button
Literal[“left”, “middle”, “right”], optional - The mouse button to release. Defaults to "left"
.report
bool, optional - Whether to include the screenshot in reporting. Defaults to True
.Image.Image
- A PIL Image object containing the screenshot.display
int, optional - The display ID to set as active.text
str - The text to be typed.typing_speed
int, optional - The speed of typing in characters per second. Defaults to 50
.message
str - The error message.operation
str - The operation that failed.error
Exception - The original error that caused the failure.message
str - The error message.timeout_seconds
float | None - Optional timeout duration in seconds.ControllerServer
for managing the AskUI Remote Device Controller process.
Handles process discovery, startup, and shutdown for the native controller binary.
clean_up
bool, optional - Whether to clean up existing processes (only on Windows) before starting. Defaults to False
.force
bool, optional - Whether to forcefully terminate the process. Defaults to False
.message
str - The error message.result
ToolResult - The ToolResult that caused the error.output
str | None, optional - The output of the tool.error
str | None, optional - The error message of the tool.base64_image
str | None, optional - The base64 image of the tool.system
str | None, optional - The system message of the tool.