> ## Documentation Index > Fetch the complete documentation index at: https://docs.askui.com/llms.txt > Use this file to discover all available pages before exploring further. # Element Detection and OCR Issues > Solutions for element selection, text recognition, and detection problems # Misspellings of Words **Problem:** The OCR model sometimes misreads characters, especially in certain fonts or noisy images. This can result in words being misclassified or misspelled, which then causes the automation to fail when it searches for exact matches. **Example:**

✅ Expected Behavior

ⵊ Text is correctly spelled:

✅ Hallo ✅

👍 Works with `click().text("Hallo")`

❌ Actual Issue

ⵊ Text is misspelled

❌ `HaII0` ❌

👎 Can't find `click().text("Hallo")`. Because of recognition issues. (`l`->`I`and `o` -> `0`)

### Solutions You can directly correct OCR predictions and improve OCR model accuracy by training your workspace-specific model. **Steps:** 1. Start the AskUI shell: ```bash theme={null} askui-shell ``` 2. Launch the OCR Teaching App: ```bash theme={null} AskUI-StartOCRTeaching ``` 3. Upload a screenshot containing the misclassified word (e.g., “Hallo”). 4. Switch to **Trained Model** for precise corrections. 5. Select the wrongly detected word (`HaII0`) and replace it with the correct label: `Hallo`. 6. Press the **Train Correction** 7. Click **"Copy Model"** to copy the newly trained model ID. 8. In your automation code, update on model config on global level or on step level to use the new model: Global Level Model Composition ```python Python theme={null} with VisionAgent(model= { "locate": ModelComposition([ModelDefinition( task="e2e_ocr", architecture="easy_ocr", version="1", interface="online_learning", useCase="", tags=["trained"] )]) }) as agent: ``` ```typescript TypeScript theme={null} // askui-helper.ts | Global Level Model Configutaion ... aui = await UiControlClient.build({ ... modelComposition: [{ "task": "e2e_ocr", "architecture": "easy_ocr", "version": "1", "interface": "online_learning", "useCase": "", "tags": ["trained"] }] }); ... ``` Step Level Model Composition ```python Python theme={null} agent.click("Alice Johnson", model=ModelComposition([ModelDefinition( task="e2e_ocr", architecture="easy_ocr", version="1", interface="online_learning", useCase="", tags=["trained"] )])) ``` ```typescript TypeScript theme={null} await aui.click().text("Alice Johnson").exec([{ "task": "e2e_ocr", "architecture": "easy_ocr", "version": "1", "interface": "online_learning", "useCase": "", "tags": ["trained"] }]) ``` # Text Detection Issues ## 1. Icon Text Merging **Problem:** Sometimes, Text Detector/annotation tool, **merges an icon and texts into one**, even though they look merged on screen. **Example:** Say you want to click **just the name** "Alice Johnson" field or **just the position** field in a interface - but OCR detects them as one long string:

✅ Expected Behavior

🖼️ Icon and Text are detected separately:

🧑 ✅ Name ✅ 🤖 ✅ Role ✅

👍 Works with `click().text("Name")` or `click().text("Name")`

❌ Actual Issue

🖼️ Icon and text are detected together:

`🧑 Name` ❌ `🤖 Role` ✅

👎 Can't find `click().text("Name")`.

### Solution You can ignore train the OCR Recognition model to ignore the OCR detection error. **Steps:** 1. Start the AskUI shell: ```bash theme={null} askui-shell ``` 2. Launch the OCR Teaching App: ```bash theme={null} AskUI-StartOCRTeaching ``` 3. Upload a screenshot containing the misclassified word (e.g., “Hallo”). 4. Switch to **Trained Model** for precise corrections. 5. Select the wrongly detected word (`HaII0`) and replace it with the correct label: `Hallo`. 6. Press the **Train Correction** 7. Click **"Copy Model"** to copy the newly trained model ID. 8. In your automation code, update on model config on global level or on step level to use the new model: Global Level Model Composition ```python Python theme={null} with VisionAgent(model= { "locate": ModelComposition([ModelDefinition( task="e2e_ocr", architecture="easy_ocr", version="1", interface="online_learning", useCase="", tags=["trained"] )]) }) as agent: ``` ```typescript TypeScript theme={null} // askui-helper.ts | Global Level Model Configutaion ... aui = await UiControlClient.build({ ... modelComposition: [{ "task": "e2e_ocr", "architecture": "easy_ocr", "version": "1", "interface": "online_learning", "useCase": "", "tags": ["trained"] }] }); ... ``` Step Level Model Composition ```python Python theme={null} agent.click("Alice Johnson", model=ModelComposition([ModelDefinition( task="e2e_ocr", architecture="easy_ocr", version="1", interface="online_learning", useCase="", tags=["trained"] )])) ``` ```typescript TypeScript theme={null} await aui.click().text("Alice Johnson").exec([{ "task": "e2e_ocr", "architecture": "easy_ocr", "version": "1", "interface": "online_learning", "useCase": "", "tags": ["trained"] }]) ``` ```python Python theme={null} agent.click("Alice Johnson", model=ModelComposition([ModelDefinition( task="e2e_ocr", architecture="easy_ocr", version="1", interface="online_learning", useCase="", tags=["word_level"] )])) ``` ```typescript TypeScript theme={null} await aui.click().text("Alice Johnson").exec([{ "task": "e2e_ocr", "architecture": "easy_ocr", "version": "1", "interface": "online_learning", "useCase": "", "tags": ["word_level"] }]) ``` ## 2. Merged Texts **Problem:** Sometimes, Text Detector/ annotation tool, **merges two separate texts into one**, even though they look clearly split on screen. **Example:** Say you want to click **just the name** "Alice Johnson" field or **just the position** field in a interface - but OCR detects them as one long string:

✅ Expected Behavior

🖼️ Text fields detected separately:

`Alice Johnson` ✅ `Software Engineer` ✅

👍 Works with `text("Alice Johnson")` or `text("Software Engineer")`

❌ Actual Issue

🖼️ Texts merged into one block:

`Alice Johnson Software Engineer`❌

👎 Can't find either one on its own.

### Solutions ```python Python theme={null} agent.click("Alice Johnson", model=ModelComposition([ModelDefinition( task="e2e_ocr", architecture="easy_ocr", version="1", interface="online_learning", useCase="00000000_0000_0000_0000_000000000000", tags=["word_level"] )])) ``` ```typescript TypeScript theme={null} await aui.click().text("Alice Johnson").exec([{ "task": "e2e_ocr", "architecture": "easy_ocr", "version": "1", "interface": "online_learning", "useCase": "00000000_0000_0000_0000_000000000000", "tags": ["word_level"] }]) ``` ```python Python theme={null} agent.click("Alice Johnson", model=ModelComposition([ModelDefinition( task="e2e_ocr", architecture="easy_ocr", version="1", interface="online_learning", useCase="", tags=["word_level"] )])) ``` ```typescript TypeScript theme={null} await aui.click().text("Alice Johnson").exec([{ "task": "e2e_ocr", "architecture": "easy_ocr", "version": "1", "interface": "online_learning", "useCase": "", "tags": ["word_level"] }]) ``` This command show how you can use an anchore element move the mouse over another element. ```tsx theme={null} await aui.moveMouseRelativeTo(100, 0).containsText("Name").exec() ``` ## 3.Text Separation **Problem:** Sometimes, Text Detector/ annotation tool, septerates a text into **two texts**, even though they look clearly merged on screen. **Example:** Say you want to click **just the name** "Alice Johnson" field or **just the position** field in a interface - but OCR detects them as two words:

✅ Expected Behavior

🖼️ Words are detected as one sentence:

`Alice Johnson` ✅

👍 Works with `text("Alice Johnson")`

❌ Actual Issue

🖼️ Words are detected as separated texts:

`Alice`❌ `Johnson`❌

👎 Can't find either `text("Alice Johnson")` on its own.

### Solution ```python Python theme={null} agent.click("Alice Johnson", model=ModelComposition([ModelDefinition( task="e2e_ocr", architecture="easy_ocr", version="1", interface="online_learning", useCase="00000000_0000_0000_0000_000000000000", tags=["word_level"] )])) ``` ```typescript TypeScript theme={null} await aui.click().text("Alice Johnson").exec([{ "task": "e2e_ocr", "architecture": "easy_ocr", "version": "1", "interface": "online_learning", "useCase": "00000000_0000_0000_0000_000000000000", "tags": ["word_level"] }]) ``` ```python Python theme={null} agent.click("Alice Johnson", model=ModelComposition([ModelDefinition( task="e2e_ocr", architecture="easy_ocr", version="1", interface="online_learning", useCase="", tags=["word_level"] )])) ``` ```typescript TypeScript theme={null} await aui.click().text("Alice Johnson").exec([{ "task": "e2e_ocr", "architecture": "easy_ocr", "version": "1", "interface": "online_learning", "useCase": "", "tags": ["word_level"] }]) ``` ## 4. Vertical Text Merging **Problem:** Sometimes, Text Detector/ annotation tool, merges two lines to one text, even though they look clearly as two lines on screen. **Example:** Say you want to click **just the name** "Alice Johnson" field or **just the position** field in a interface - but OCR detects them as one:

✅ Expected Behavior

🖼️ Texts are detected as two lines:

`Alice Johnson` ✅

👍 Works with `text("Alice Johnson")`

❌ Actual Issue

🖼️ Texts are detected as one text:

``❌

👎 Can't find either `text("Alice Johnson")` on its own.

### Solution ```python Python theme={null} agent.click("Alice Johnson", model=ModelComposition([ModelDefinition( task="e2e_ocr", architecture="easy_ocr", version="1", interface="online_learning", useCase="00000000_0000_0000_0000_000000000000", tags=["word_level"] )])) ``` ```typescript TypeScript theme={null} await aui.click().text("Alice Johnson").exec([{ "task": "e2e_ocr", "architecture": "easy_ocr", "version": "1", "interface": "online_learning", "useCase": "00000000_0000_0000_0000_000000000000", "tags": ["word_level"] }]) ``` ```python Python theme={null} agent.click("Alice Johnson", model=ModelComposition([ModelDefinition( task="e2e_ocr", architecture="easy_ocr", version="1", interface="online_learning", useCase="", tags=["word_level"] )])) ``` ```typescript TypeScript theme={null} await aui.click().text("Alice Johnson").exec([{ "task": "e2e_ocr", "architecture": "easy_ocr", "version": "1", "interface": "online_learning", "useCase": "", "tags": ["word_level"] }]) ``` ## 5. Single Character not Detected **Problem:** Sometimes, Text Detector/ annotation tool, does not detect single charactors, even though they look clearly on screen. **Example:** Say you want to click \*\*just the character "2" - but OCR does not detects them:

✅ Expected Behavior

🖼️ Single chars are detected:

`1` ✅ `2` ✅ `3` ✅

👍 Works with `text("2")`

❌ Actual Issue

🖼️ Char 2 is not detected:

`1` ✅ `2` ❌ `3` ✅

👎 Can't find either `text("2")` on its own.

### Solution Single characters are sometimes flaky. So it's better to relay on AI element. **Steps:** 1. Open AskUI Shell ```bash theme={null} askui-shell ``` 2. Create a new AI Element ```bash theme={null} # Capture elements from your screen AskUI-NewAiElement -Name "my-element-name" ``` 3. Use captured AI Elements in your code: ```python Python theme={null} from askui import locators as loc ... with VisionAgent() as agent: agent.click(loc.AiElement("my-element-name")) ``` ```python Typescript theme={null} await aui.click().aiElement("my-element-name").exec() ``` If you cannot use the AskUI-NewAIElement command, activate experimental commands by running `AskUI-ImportExperimentalCommands` in your terminal. ## 6. Text not Detected **Problem:** Sometimes, for no apparent reason, Text Detector/ annotation tool does not detect a text, even though you can see it clearly on screen. **Example:** Say you want to click **just the name** "Alice Johnson" field - but OCR does not detects the text at all:

✅ Expected Behavior

🖼️ Text was detected:

`Alice Johnson` ✅

👍 Works with `text("Alice Johnson")`

❌ Actual Issue

🖼️ Text wasn't detected

`Alice Johnson`❌

👎 Can't find either `text("Alice Johnson")` on its own.

### Solution In the case the text was not detected you have to use the AI Element. **Steps:** 1. Open AskUI Shell ```bash theme={null} askui-shell ``` 2. Create a new AI Element ```bash theme={null} # Capture elements from your screen AskUI-NewAiElement -Name "my-element-name" ``` 3. Use captured AI Elements in your code: ```python Python theme={null} from askui import locators as loc ... with VisionAgent() as agent: agent.click(loc.AiElement("my-element-name")) ``` ```python Typescript theme={null} await aui.click().aiElement("my-element-name").exec() ``` If you cannot use the AskUI-NewAIElement command, activate experimental commands by running `AskUI-ImportExperimentalCommands` in your terminal. # Element Detection Issues ## 7. Checkbox not Detected **Problem:** The element detection model does not detect checkboxes as interactive elements. This means you cannot directly target a checkbox using element detection. **Example:**

✅ Expected Behavior

🖼️ Checkbox is detected as an element:

❌ Actual Issue

🖼️ Checkbox is not detected:

### Solutions Instead of clicking the checkbox directly, use a nearby text label as an anchor and click relative to it. ```python Python theme={null} from askui import locators as loc ... with VisionAgent() as agent: agent.click(loc.RelativePosition(x=-30, y=0, anchor=loc.Text("Accept Terms"))) ``` ```python Typescript theme={null} await aui.moveMouseRelativeTo(-30, 0).text("Accept Terms").exec() await aui.mouseLeftClick().exec() ``` Adjust the relative offset values (`x`, `y`) based on the position of the checkbox relative to the anchor text in your application. You can teach AskUI to recognize the checkbox by creating a custom AI Element. **Steps:** 1. Open AskUI Shell ```bash theme={null} askui-shell ``` 2. Create a new AI Element ```bash theme={null} # Capture elements from your screen AskUI-NewAiElement -Name "my-element-name" ``` 3. Use captured AI Elements in your code: ```python Python theme={null} from askui import locators as loc ... with VisionAgent() as agent: agent.click(loc.AiElement("my-element-name")) ``` ```python Typescript theme={null} await aui.click().aiElement("my-element-name").exec() ``` If you cannot use the AskUI-NewAIElement command, activate experimental commands by running `AskUI-ImportExperimentalCommands` in your terminal.