Skip to main content
Version: 0.18.0

Cannot Find Element Error

If you see this error message when you run AskUI you have a few different options you can try to resolve it. We will go over all of them on this page:

# Executed instruction:
await aui.clickButton({label: 'I feel happy!'});

# Error message
A problem occurred while executing the instruction: 
Click on button with text "I feel happy!".
Reason ControlCommandError: Cannot find "button contains with text 'I feel happy!'"

Use Interactive Annotation to Debug​

When you develop an automation you can use interactive annotation to extract filters as described on our interactive annotation page.

This animated image illustrates the process: Interactive Annotation in action

If an element-description derived through this method does not work you may run the interactive annotation again and check if AskUI gives you the same element-description. Sometimes elements are hard to identify correctly, like icons or specific fonts. Read the next sections on how to tackle this 🙂

Selecting Icons​

There are millions of different icon sets out there. Detecting them all reliable is nearly impossible.

You can use three strategies to make your instructions more stable.

1. Use Relative Element-Descriptors​

Often there is text near a specific icon. You can use this to your advantage, like in the following example, where the icon below a specific text is clicked. See our guide about relational element-descriptors to understand how those work:

await aui
  .click()
  .icon()
  .below()
  .text() 
  .withText('Please Enter your Name').exec();

2. Use Generic Element-Descriptor​

It could be the case that the target element is not detected correctly, e.g, an icon could possibly be detected as a toggle or checkbox and vice versa. In such cases, the generic element-descriptor element() could be a good option.

Be aware that element() alone specifies no particular property. It is recommended to be used in conjunction with a relational element descriptor:

await aui
  .click()
  .element()
  .below()
  .text()
  .withText('Please Enter your Name').exec();

This method is not restricted to icon elements, but can also be used for a wide range of elements.

3. Use Custom Element Selection​

Custom element selection is your last resort because it is slow as it uses image-in-image search. Check out our Custom Elements page to see how you can make use of it.

Selecting Image​

Images are supposed to have a very certain visual property.

  • An image is expected to have a color contrasting against the background.
  • An image is expected to have a rectangular shape. (rounded corner in a certain degree can be accepted)

Generally speaking, everything on the screen can be an image. For example, icons on a website are actually images. But in terms of visual semantics, we don't deal with the notion of image in general. So it is worth knowing that the above two conditions are critical when using the image() descriptor.

Selecting Text​

OCR is tricky and we are working hard to improve it. But you will encounter cases where some text may not be recognized reliably depending on the font size, style or typeface:

1. Missing Blankspaces Between Words​

You can guard against missing blankspaces with withTextRegex():

// Use [\\s]{0,1} as a replacement for whitespace
await aui
  .click()
  .text()
  .withTextRegex('your[\\s]{0,1}name').exec();

2. Cut off characters at the start/end​

For cut-off characters you can omit the first/last character:

// Omit first and last character of 'your name'
await aui
  .click()
  .text()
  .withText('your name').exec();

Missing Permissions for AskUI Controller on macOS​

AskUI controls your operating system and macOS is very restrictive in giving access to third-party applications (which is a good thing actually!). The AskUI Controller needs to be given access for Screen Recording and Accessibility. See our troubleshooting macOS page.