Configure Different Models

This guide shows you how to configure and use different AI models with AskUI to optimize performance for your specific automation tasks. For a deeper understanding of how these models work and their capabilities, see AI Models.

Overview

AskUI supports multiple AI model providers, each with different strengths:

AskUI Models: Fast, production-ready models optimized for UI automation
Anthropic Models: Advanced language models for complex reasoning tasks
Hugging Face Models: Open-source models for experimentation
Self-Hosted Models: Custom models you host yourself

Quick Reference

Platform	Available Models
AskUI	`askui`, `askui-combo`, `askui-pta`, `askui-ocr`, `askui-ai-element`
Anthropic	`anthropic-claude-3-5-sonnet-20241022`
Hugging Face	`AskUI/PTA-1`, `OS-Copilot/OS-Atlas-Base-7B`, `showlab/ShowUI-2B`, `Qwen/Qwen2-VL-2B-Instruct`, `Qwen/Qwen2-VL-7B-Instruct`
Self-Hosted	`UI-Tars`

For detailed specifications of all available AI models, see the AI Models Reference.

Step 1: Choose Your Model Provider

First, decide which model provider best fits your needs:

When to Use AskUI Models

Production environments: Fast, reliable, and enterprise-ready
Standard UI automation: Optimized for clicking, typing, and data extraction
Cost-conscious projects: Lower cost per operation

When to Use Anthropic Models

Complex reasoning tasks: Advanced decision-making capabilities
Natural language interactions: Better understanding of complex instructions
Experimental projects: Cutting-edge AI capabilities

When to Use Hugging Face Models

Open-source requirements: Community-driven development
Research and experimentation: Access to latest research models
Budget constraints: Free tier available (rate-limited)

Step 2: Set Up Authentication

Configure authentication for your chosen model provider:

AskUI Models
Anthropic Models
Hugging Face Models

AskUI models require workspace credentials from your AskUI account.Required Environment Variables:

ASKUI_WORKSPACE_ID
ASKUI_TOKEN

Get Your Credentials:

Sign in to AskUI Hub
Navigate to your workspace settings
Copy your Workspace ID and generate an access token

Anthropic models require an API key from Anthropic.Required Environment Variables:

ANTHROPIC_API_KEY

Get Your API Key:

Sign up at console.anthropic.com
Navigate to API Keys
Generate a new API key

Step 3: Configure Environment Variables

Set the required environment variables for your operating system:

Linux & macOS
Windows PowerShell
Windows CMD

# For AskUI models
export ASKUI_WORKSPACE_ID=<your-workspace-id>
export ASKUI_TOKEN=<your-access-token>

# For Anthropic models
export ANTHROPIC_API_KEY=<your-api-key>

Add these to your ~/.bashrc or ~/.zshrc file to persist across sessions.

# For AskUI models
$env:ASKUI_WORKSPACE_ID="<your-workspace-id>"
$env:ASKUI_TOKEN="<your-access-token>"

# For Anthropic models
$env:ANTHROPIC_API_KEY="<your-api-key>"

Use [Environment]::SetEnvironmentVariable() to set permanent system variables.

REM For AskUI models
set ASKUI_WORKSPACE_ID=<your-workspace-id>
set ASKUI_TOKEN=<your-access-token>

REM For Anthropic models
set ANTHROPIC_API_KEY=<your-api-key>

Step 4: Use Models in Your Code

Specify which model to use by adding the model parameter to your commands:

Basic Model Usage

from askui import VisionAgent

with VisionAgent() as agent:
    # Use AskUI's default model (recommended for most cases)
    agent.click("login button")
    
    # Use a specific AskUI model
    agent.click("search field", model="askui-ocr")
    
    # Use Anthropic's model for complex reasoning
    agent.act("Fill out this form with realistic test data", model="anthropic-claude-3-5-sonnet-20241022")

Model Selection Strategy

Choose models based on your task requirements:

# For simple element clicking - use fast AskUI models
agent.click("submit button", model="askui-pta")

# For text extraction - use OCR-optimized models
text = agent.get("user name field", model="askui-ocr")

# For complex multi-step tasks - use reasoning models
agent.act("Navigate to settings and enable dark mode", model="anthropic-claude-3-5-sonnet-20241022")

# For experimental features - try Hugging Face models
agent.click("menu icon", model="AskUI/PTA-1")

Step 5: Verify Your Configuration

Test your model configuration with a simple script:

from askui import VisionAgent

def test_model_configuration():
    with VisionAgent() as agent:
        try:
            # Test AskUI model
            agent.click("desktop", model="askui")
            print("✓ AskUI models configured correctly")
            
            # Test Anthropic model (if configured)
            agent.act("Take a screenshot", model="anthropic-claude-3-5-sonnet-20241022")
            print("✓ Anthropic models configured correctly")
            
        except Exception as e:
            print(f"✗ Configuration error: {e}")

if __name__ == "__main__":
    test_model_configuration()

Troubleshooting

Authentication errors

Verify environment variables are set correctly
Check for extra spaces in API keys
Ensure API keys are valid and not expired
Restart your terminal/IDE after setting variables

Model not found errors

Check model name spelling (case-sensitive)
Verify the model is supported for your command
Some models may not support all AskUI commands

Rate limiting issues

Hugging Face models have rate limits
Consider upgrading to paid tiers for higher limits
Implement retry logic with exponential backoff

Next Steps

Now that you have models configured:

Optimize Performance: Learn about model selection best practices
Advanced Usage: Explore agentic workflows
Production Deployment: Review enterprise considerations

Documentation

Tutorial

How-to Guides

Understanding AskUI

Additional Resources

Configure Different Models

Overview

Quick Reference

Step 1: Choose Your Model Provider

When to Use AskUI Models

When to Use Anthropic Models

When to Use Hugging Face Models

Step 2: Set Up Authentication

Step 3: Configure Environment Variables

Step 4: Use Models in Your Code

Basic Model Usage

Model Selection Strategy

Step 5: Verify Your Configuration

Troubleshooting

Next Steps

​Overview

​Quick Reference

​Step 1: Choose Your Model Provider

​When to Use AskUI Models

​When to Use Anthropic Models

​When to Use Hugging Face Models

​Step 2: Set Up Authentication

​Step 3: Configure Environment Variables

​Step 4: Use Models in Your Code

​Basic Model Usage

​Model Selection Strategy

​Step 5: Verify Your Configuration

​Troubleshooting

​Next Steps

Overview

Quick Reference

Step 1: Choose Your Model Provider

When to Use AskUI Models

When to Use Anthropic Models

When to Use Hugging Face Models

Step 2: Set Up Authentication

Step 3: Configure Environment Variables

Step 4: Use Models in Your Code

Basic Model Usage

Model Selection Strategy

Step 5: Verify Your Configuration

Troubleshooting

Next Steps