Single-Step Actions in AskUI

AskUI provides a comprehensive set of single-step actions that allow you to interact with any UI element on your screen. These actions range from basic mouse clicks to complex keyboard combinations and system operations.

Core Interaction Commands

DescriptionModelsExample (Python)
click()Clicks on an element described by textAllagent.click('Login button')
type()Types text into a focused elementAllagent.type('username@example.com', "Username input field")
mouse_move()Moves the mouse over located elementAllagent.mouse_move('red button')

Detailed usage on how to use click can be found here.

Tools

AskUI provides several built-in tools to interact with the operating system and applications:

DescriptionExample (Python)
osProvides OS-level operationsagent.tools.os.keyboard_release()
webbrowserControls web browser operationsagent.tools.webbrowser.open_new("https://askui.com")
clipboardManages clipboard operationsagent.tools.clipboard.copy("Text to copy")

Assertion and Information Commands

DescriptionModelsExample (Python)
get()Extracts text or information from the screensonnet-3.5-latesttext = agent.get('What is the value in the total field?')