AskUIDocs
Extending

Your own model — AskUI AirGap and ModelBridge

Point AskUI Desktop at your own model, on AskUI AirGap Hosting or through the AskUI ModelBridge (Azure AI Foundry, Vertex AI, any Anthropic- or OpenAI-compatible API), instead of the AskUI hub.

By default, AskUI Desktop runs inference through the AskUI hub, billed to your workspace, nothing to configure. Bring your own model (BYOM) points the agent at a model you control instead: on your AskUI AirGap Hosting, or through the AskUI ModelBridge — your model from Azure AI Foundry, Vertex AI or any compatible API. Pick where the model runs, connect to it, then name the model.

Note

BYOM changes only where inference runs, the agent, prompts, tools, and run reports are identical.

Where to configure it

Configure it under Settings → Account → AskUI Model, switch an already‑running install to BYOM, or edit/remove an existing configuration. The same form appears in the License setup wizard and wherever a feature has its own model (Forge, Dreaming, In-Flight Guidance).

Choose where your model runs

The form looks the same everywhere. First pick where the model runs:

  • AskUI Inference: AskUI's hosted models, billed to your workspace, nothing to set up. Requires signing in to an AskUI workspace.
  • AskUI AirGap: a model on your AskUI AirGap Hosting, inside your own network.
  • AskUI ModelBridge: your own model from a cloud account or any compatible API. Its providers (Azure AI Foundry, Vertex AI, Other; AWS Bedrock coming soon) show in a second row, Model Provider, once you pick it.

What your mode cannot use is shown as Not available, with the reason:

AskUI InferenceAskUI AirGapAskUI ModelBridge
License modeNot available (sign in to AskUI)✓✓
Signed in to AskUI✓Not available (License only)Not available (License only)

To run on your own model, use a license key. An AirGap or ModelBridge setup you saved in License mode is kept, but not used while you are signed in; your runs use AskUI Inference instead.

A model on your AskUI AirGap Hosting, inside your own network: screenshots and prompts never leave it.

FieldRequiredNotes
Base URLRequiredThe base URL of your AskUI AirGap Hosting, e.g. http://airgap.internal:8000/v1.
API keyOptionalOnly if your hosting requires one (always required for the Anthropic API).

A model deployed in your Azure AI Foundry resource, Claude and OpenAI-compatible models alike.

FieldRequiredNotes
EndpointRequiredYour resource's full endpoint: ending in /anthropic for Claude models, in /openai/v1 for the others.
API keyRequiredThe key of your Foundry resource.

Claude, Gemini and other models served by Vertex AI from your own Google Cloud project: Google bills the project, and requests are processed in the location you choose.

FieldRequiredNotes
Service account keyOptionalThe JSON key of a service account with the Vertex AI User role (roles/aiplatform.user), from IAM & Admin → Service accounts → Keys. If your organisation forbids keys, choose the file gcloud auth application-default login writes instead. Without a file, this computer's Google login is used.
Google Cloud project IDRequiredFilled in from the key file.
LocationRequiredeu or us (multi-region: data stays in that geography), global (wherever Google has capacity), or a single region such as europe-west4. Newer Claude models run on eu, us or global; Gemini on global or a single region.

Enable each model for your project in the Vertex AI Model Garden first, and request quota for it in the location you pick. Model ids are Vertex's own: claude-sonnet-5-5 (older Claude models carry a date, claude-opus-4-5@20251101), and Gemini with its publisher, google/gemini-2.5-flash.

Any Anthropic‑ or OpenAI‑compatible API: Anthropic or OpenAI directly, OpenRouter, or your own gateway.

FieldRequiredNotes
Base URLRequired, except for AnthropicLeave blank with a Claude model for Anthropic's own API. An OpenAI-compatible URL ends in /v1, e.g. https://openrouter.ai/api/v1.
API keyRequired for the Anthropic APIGenerate an Anthropic key in the Anthropic console. Optional for an OpenAI-compatible server that doesn't require one.

Authentication has a second mode for a gateway fronted by Microsoft Entra ID: e.g. an internal LLM gateway of your company. Choose Microsoft Entra (OAuth2 client credentials) instead of API key (or none) and fill in the app registration:

  • Tenant ID: GUID or domain, e.g. contoso.onmicrosoft.com (required)
  • Client ID: the application (client) id of your app registration (required)
  • Scope: the resource scope, usually ending in /.default, e.g. api://<app-id>/.default (required)
  • Client secret: stored encrypted on this device, like an API key (required)

Saving acquires a token to verify the app registration; during runs, tokens are minted and refreshed automatically.

AWS Bedrock shows as Coming soon: it authenticates differently (SigV4) and is not wired up yet. Until then, put a gateway in front that re-exposes it Anthropic- or OpenAI-compatibly and use Other, or talk to us.

Name the model

Enter the provider's Model id, e.g. claude-sonnet-5-5 or Qwen/Qwen3-VL-235B-A22B-Instruct. It is required.

The protocol is picked for you, and the form shows which one and why:

  • a model id containing claude uses the Anthropic Messages API,
  • unless the base URL ends in /v1: that is an OpenAI-compatible gateway, so a Claude model on OpenRouter (https://openrouter.ai/api/v1) uses the OpenAI-compatible API,
  • every other model uses the OpenAI-compatible Chat Completions API,
  • on Azure AI Foundry the endpoint decides, whatever your deployment is called: one ending in /anthropic is the Anthropic API, anything else OpenAI-compatible,
  • on Vertex AI a Claude model uses Vertex's Anthropic API and every other model (Gemini) its OpenAI-compatible endpoint; there is nothing to change.

If the form can't tell, e.g. a Claude model behind a gateway whose URL doesn't end in /v1, select Change under the model and pick the protocol yourself.

OpenAI-compatible models: experimental, no support provided

Models on the OpenAI-compatible protocol are experimental. They may work with capable vision models, but we do not test against them and cannot offer support for issues specific to this configuration.

The agent drives a UI from screenshots, so it needs a vision‑capable model. A text‑only model will connect but won't be able to see the screen.

Below the model: Model rates (your provider's prices, for the cost shown in run reports), Image handling and, for the test-run model, Truncation.

Supported models

  • Claude models (recommended): the Anthropic Messages API is the tested path, whether you reach it through the AskUI hub, your own Anthropic key, Azure AI Foundry, or an Anthropic-compatible gateway. Some internal gateways authenticate differently: either put something in front that re-exposes them as an Anthropic- or OpenAI-compatible endpoint, or talk to us and we help you get your authentication scheme supported. Any current Claude model with vision works; ids come from Anthropic's model list. Newer Claude generations also use adaptive thinking automatically.
  • GPT-5.4 models: supported through the OpenAI-compatible protocol: gpt-5.4, gpt-5.4-mini and gpt-5.4-nano, from OpenAI directly or via OpenRouter. They see screenshots and call tools, so they drive a UI, they're just not the path AskUI tests against.
  • Gemini vision models: gemini-2.5-pro, gemini-3-pro-preview and the Flash variants work the same way: on the Vertex AI tab (google/gemini-2.5-flash), or through any gateway that exposes them OpenAI-compatibly (OpenRouter, or your own proxy).
  • Self-hosted VLMs: a screen-capable model such as Qwen/Qwen3-VL-… on AskUI AirGap Hosting, which keeps screenshots inside your own network.
  • What won't work: text-only models, and models without tool/function calling. Every action the agent takes is a tool call on a screenshot; without both it cannot execute a step.

Model choice is the main lever on run cost and duration, the trade-offs and what to watch: Performance & cost.

Add a provider

Open Settings → Account → AskUI Model (or reach it via license activation).

Pick where your model runs, fill in the connection fields per the tables above, then enter the Model, the protocol follows from it.

Select Save. AskUI Desktop makes a test call to the endpoint before saving, if the key, URL, or model is wrong, you'll see the error here rather than mid‑run.

Run a test as usual. The agent now uses your provider.

Behind a corporate proxy, configure network access first, the setup wizard's Behind a proxy? Configure network access section, or Settings → Network proxy: otherwise the test call can't reach the endpoint. Details: Network proxy.

Gateway message transforms

Some corporate gateways enforce rules stricter than the stock OpenAI spec, and reject the agent's requests with HTTP 400 even though the model behind them would accept them. For these, an operator project can normalize the outgoing messages array: drop a class implementing IOpenAiMessageTransform into utils/message_transforms/.

utils/message_transforms/gateway_normalization.cs
public sealed class GatewayNormalization : IOpenAiMessageTransform
{
    public JsonArray Transform(JsonArray messages)
    {
        var output = new JsonArray();
        foreach (var node in messages)
        {
            var message = (JsonObject)node!.DeepClone();
            // Example gateway rule: an assistant message carrying
            // tool_calls must have content = null.
            if ((string?)message["role"] == "assistant"
                && message["tool_calls"] is JsonArray { Count: > 0 })
            {
                message["content"] = null;
            }
            output.Add(message);
        }
        return output;
    }
}

Transform files compile at run start exactly like custom tools, no using directives needed, a compile error aborts the run, and the same rule applies: developer territory. Multiple files compose in discovery order. Only the OpenAI-compatible provider consults them; the app ships no gateway-specific rules, your project owns them.

How your credentials are stored

  • The API key is stored encrypted on this device (on Windows, via DPAPI scoped to your user account). The non‑secret fields, where the model runs, protocol, model id, base URL, are stored in plain local settings.
  • The key is passed to the run in memory and is never written to the run report or logs.
  • Editing other fields won't wipe a saved key, leave the key field blank to keep the existing one.
  • A saved key belongs to the place it was saved for. Switch to another tab and the key field is empty; save there without a new key and the old key is removed, so it is never sent to a different host.

Switching back to the AskUI hub

In Settings → Account → AskUI Model, choose Remove key. This reverts to the AskUI hub default and forgets the stored key. (In License mode there is no hub default, so you replace the provider rather than removing it.)

On this page