Learn how to interact with UI elements using AskUI’s powerful tools and actions
Once you’ve selected an element, you can interact with it using AskUI’s comprehensive set of actions and tools. This guide covers the different ways to interact with elements, from basic clicks to complex keyboard operations.
Available interaction methods vary depending on the agent type you’re using. Desktop agents support different interactions than mobile agents.
from askui import VisionAgentwith VisionAgent() as agent: # Click actions agent.click("Login button") # Single click - see click() API reference agent.click("Submit", button="right") # Right click agent.click("Open", button="middle") # Middle click # Text input agent.type("myuser@example.com") # Type text # Mouse movement agent.mouse_move("Menu item") # Move to element agent.mouse_move(100, 200) # Move to coordinates
from askui import VisionAgentwith VisionAgent() as agent: # Open new browser window agent.tools.webbrowser.open_new("https://askui.com") # Open in new tab agent.tools.webbrowser.open_new_tab("https://docs.askui.com")