AI Model Usage
Learn how to use and configure different AI models
What models is AskUI using?
AskUI uses a layered system of AI models, each with a distinct role in understanding, executing, and interacting with user interfaces. Here’s how we classify and use them:
-
Grounding Models (Locators)
- Grounding models identify and interact with UI elements on the screen.
-
Query Models (Asks)
- Responsible for answering user queries or generating intelligent responses.
-
Large Action Models (act command) (Multi Step)
- Responsibilites
- Goal to → Planning
- Delegate Grounding Models
- Delegate Query Models
- Reflection of Errors
- UI-Tars
- Computer-Use
Model Type Model Name Purpose Teachable Online Trainable Grounding UIDT-1 Locate elements & understand screen No Partial Grounding PTA-1 Convert prompts into one-click actions No Yes Query GPT-4 Understand & respond to user queries Yes No Query Computer Use Understand & respond to user queries Yes No Large Action (act) Computer Use Plan and execute full workflows Yes No Large Action (act) UI-Tars Plan and execute full workflows Yes No - Responsibilites