Skip to main content

2024.09.30 | AskUI Suite 24.9.1

· 2 min read
Johannes Dienst
Developer Advocate

Introduction

This release delivers version 24.9.1 of the AskUI Suite.


New Features


Fixes

  • Fixed a compatibility issue that caused a crash on older macOS versions due to the usage of C++20 functionality.

Improvements

  • Windows installer is signed with the Microsoft provided certificate.
  • Android devices can now be address by their device id:
  • Windows and Linux ARM64 support.
  • Error reporting commandlet collects software and hardware information about the system.
  • Controller and Snipping Tool logs can be configured through a configuration file: See docs
  • Our GUID is now RFC-4122 version 4 compliant.
  • Local proxy settings are applied to npm.
  • macOS and Linux installers can be used in silent mode. See docs

Supported Operating Systems

  • Windows 11
    • Version: 22H2
    • Architecture(s): AMD64 and ARM64
  • MacOS
    • Version: 14.x
    • Architecture(s): ARM64 (Apple Silicon)
  • Linux
    • Version: Ubuntu 22.04.2
    • Architecture(s): AMD64 and ARM64

Windows Installation

AMD64 Download link.

ARM64 Download link.

macOS Installation

# Download installer into /tmp folder
curl -o /tmp/AskUI-Suite-24.9.1-User-Installer-MacOS-ARM64-Full.run https://files.askui.com/releases/Installer/24.9.1/AskUI-Suite-24.9.1-User-Installer-MacOS-ARM64-Full.run

# Run the installer
bash /tmp/AskUI-Suite-24.9.1-User-Installer-MacOS-ARM64-Full.run

Linux Installation

# Download installer into /tmp folder (AMD64)
curl -o /tmp/AskUI-Suite-24.9.1-User-Installer-Linux-x64-Full.run https://files.askui.com/releases/Installer/24.9.1/AskUI-Suite-24.9.1-User-Installer-Linux-x64-Full.run

# Run the installer
bash /tmp/AskUI-Suite-24.9.1-User-Installer-Linux-x64-Full.run

# Download installer into /tmp folder (ARM64)
curl -o /tmp/AskUI-Suite-24.9.1-User-Installer-Linux-x64-Full.run https://files.askui.com/releases/Installer/24.9.1/AskUI-Suite-24.9.1-User-Installer-Linux-ARM64-Full.run

# Run the installer
bash /tmp/AskUI-Suite-24.9.1-User-Installer-Linux-ARM64-Full.run

Versions

Components and Packages

  • AskUI Controller
    • AskUI Remote Device Controller: 0.11.0.0
    • AskUI Legacy UI Controller: 0.14.7
    • AskUI Remote Device Snipping Tool: 0.2.0.0
  • AskUI Runner: 0.1.4
  • AskUI Development Environment (ADE): 0.9.1.0
  • AskUI Development Kit (ADK): 0.20.7
  • AskUI OCR Teaching: 0.4.0

3rd Party Versions

  • Node.JS: 21.0.0
  • PowerShell 7.3.12
  • PxProxy: 0.9.2

2024.08.12 | AskUI Suite 24.7.1

· 2 min read
Johannes Dienst
Developer Advocate

Introduction

This release delivers version 24.7.1 of the AskUI Suite, which includes debug rendering features, support for local proxies and non-admin Windows Installation.


New Features

  • Debug rendering for internal use.
    • Animated mouse movement vectors.
    • Scrollable text view of executed controller actions (and their parameters; not ADK steps).
    • Step index and icon of executed action on the screen (non-mouse actions are placed at the current mouse position).
  • Support for local proxies.
  • Non-admin Windows installation.

Changes

  • The AskUI Controller logs into ~/.askui/Logs/AskUI Controller directory.
  • The execOnShell ADK function supports specifying a timeout.
  • Status indicator rendering is on by default now.

Fixes

  • AskUI-NewProject commandlet does not copy user settings to project settings anymore.

Improvements

  • Overlay rendering should require less resources (CPU/GPU).

Supported Operating Systems

  • Windows 11
    • Version: 22H2
    • Architecture(s): AMD64 and ARM64
  • MacOS
    • Version: 14.x
    • Architecture(s): ARM64 (Apple Silicon)
  • Linux
    • Version: Ubuntu 22.04.2
    • Architecture(s): AMD64

Windows Installation

Download link.

macOS Installation

# Download installer into /tmp folder
curl -o /tmp/AskUI-Suite-24.7.1-User-Installer-MacOS-ARM64-Full.run https://files.askui.com/releases/Installer/24.7.1/AskUI-Suite-24.7.1-User-Installer-MacOS-ARM64-Full.run

# Run the installer
bash /tmp/AskUI-Suite-24.7.1-User-Installer-MacOS-ARM64-Full.run

Linux Installation

# Download installer into /tmp folder
curl -o /tmp/AskUI-Suite-24.7.1-User-Installer-Linux-x64-Full.run https://files.askui.com/releases/Installer/24.7.1/AskUI-Suite-24.7.1-User-Installer-Linux-x64-Full.run

# Run the installer
bash /tmp/AskUI-Suite-24.7.1-User-Installer-Linux-x64-Full.run

Versions

Components and Packages

  • AskUI Controller
    • AskUI Remote Device Controller: v0.9.0.0
    • AskUI Legacy UI Controller: 0.14.6
    • AskUI Remote Device Snipping Tool: v0.1.1.0
  • AskUI Runner: 0.1.4
  • AskUI Development Environment (ADE): v0.9.0.0
  • AskUI Development Kit (ADK): 0.20.7

3rd Party Versions

  • Node.JS: 21.0.0
  • PowerShell 7.3.12

2024.08.05 | ADK 0.20.7 - macOS and Linux Installer

· 4 min read
Johannes Dienst
Developer Advocate

Introduction

This release delivers some improvements to the AskUI initial setup and the error reporting.

  • Video recording is disabled by default now.
  • We added a .gitignore file to the initial project. This prevents checking in unwanted files like reports or .env.
  • If an instruction fails the error log contains the code including the line number, so you can find the failing AskUI Instruction fast.

We also release the macOS and Linux Installer which brings the AskUI Suite to macOS and Linux. It includes the ADE and the new implementation of the AskUI Controller. The ADE makes managing all the AskUI components and AskUI projects way easier and streamlined. While the new AskUI Controller improves the execution speed and operating system compatibility. A better developer experience and faster execution for you!


New Features

Video Recording Is Disabled by Default

Video recording is disabled by default in helpers/askui-helper.ts. Enable it by uncommenting the commented-out lines.

Relevant lines in helpers/askui-helper.ts
...
beforeEach(async () => {
/* Uncomment to enable video recording
await aui.startVideoRecording();
*/
});

afterEach(async () => {
/* Uncomment to enable video recording
await aui.stopVideoRecording();
const video = await aui.readVideoRecording();
await AskUIAllureStepReporter.attachVideo(video);
*/
});
...

.gitignore for Reports-Directories and Configuration Files

When you create a new project a sensible .gitignore file will be added. So you do not check in the following files accidentally:

allure-results/
report/
xray-report/

.env
.vscode

# Dangerous (may contain secrets)
.npmrc

# Dependencies
/node_modules

# Caches
.npm
.eslintcache

# Misc
.DS_Store

# Logs
npm-debug.log*
yarn-debug.log*
yarn-error.log*

Helpful Error Message When AskUI Instruction Fails

The error messages Jest displayed when an AskUI Workflow/Instruction failed made it hard to find the offending line of code. Especially on larger workflows, it was a challenge. Now the error message contains the exact line and also the code that failed:

The image is a screenshot of a Jest test failure log from a test file named my-first-askui-test-suite.test.ts. The log shows an error message indicating a ControlCommandError due to the inability to find a button containing the text "aloha blub". The error occurred in an asynchronous test function intended to "produce an error log". The relevant code snippet highlights the problematic line with await aui.click().button().withText('aloha blub').exec();. The stack trace provides details of where the error occurred within the test file. There is also a reference to documentation for troubleshooting this issue.


AskUI Installer Update 24.6.2

This ships the AskUI Controller and the AskUI Suite for all major desktop operating systems.

Since the support for MacOS and Linux is new there might be undiscovered issues that will be tackled step by step.

Supported Operating Systems

  • Windows 11
    • Version: 22H2
    • Architecture(s): AMD64 and ARM64
  • MacOS
    • Version: 14.x
    • Architecture(s): ARM64 (Apple Silicon)
  • Linux
    • Version: Ubuntu 22.04.2
    • Architecture(s): AMD64

Windows Installation

Download link.

macOS Installation

# Download installer into /tmp folder
curl -o /tmp/AskUI-Suite-24.6.2-User-Installer-MacOS-ARM64-Full.run https://files.askui.com/releases/Installer/24.6.2/AskUI-Suite-24.6.2-User-Installer-MacOS-ARM64-Full.run

# Run the installer
bash /tmp/AskUI-Suite-24.6.2-User-Installer-MacOS-ARM64-Full.run

Linux Installation

# Download installer into /tmp folder
curl -o /tmp/AskUI-Suite-24.6.2-User-Installer-Linux-x64-Full.run https://files.askui.com/releases/Installer/24.6.2/AskUI-Suite-24.6.2-User-Installer-Linux-x64-Full.run

# Run the installer
bash /tmp/AskUI-Suite-24.6.2-User-Installer-Linux-x64-Full.run

New Features

Support for macOS and Linux.


Changes

  • The status indicator of the AskUI Controller was changed to match the users expectation.
    • Stats that are rendered:
      • Workflow in Progress
        • Color: Dark blue (Logo color)
        • Description: A workflow is in progress but the AskUI Controller waits for the next action.
      • Execution
        • Color: Green
        • Description: Indicates that for instance a mouse move or typing is in progress.
      • Idle
        • Color: White
        • Description: AskUI Controller is running but no workflow is in progress.

Fixes

  • N&A

Versions

Components and Packages

  • AskUI Controller
    • AskUI Remote Device Controller: v0.9.0.0
    • AskUI Legacy UI Controller: 0.14.3
    • AskUI Remote Device Snipping Tool: v0.1.1.0
  • AskUI Runner: 0.1.4
  • AskUI Development Environment (ADE): v0.9.0.0
  • AskUI Development Kit (ADK): 0.20.7

3rd Party Versions

  • Node.JS: 21.0.0
  • PowerShell 7.3.12

2024.07.01 | Relational Selector - Intersecting Option

· One min read
Johannes Dienst
Developer Advocate

Introduction

This release adds a new parameter intersection_area to the relational selectors above(), below(), rightOf() and leftOf(). It allows you to specify more granular which elements get detected.


New Features

Optional parameter intersection_area for relational selectors above(), below(), rightOf() and leftOf() in AskUI TypeScript SDK:


Fixes

  • Drag & Drop (Mouse down, mouse move, mouse up) crashes controller.
  • TypeScript ADK retries requests to AskUI Inference that need fixing by a user. Resulted in long execution times because of unsuccessful retries.
  • AskUI Controller crashes when trying to implement Drag & Drop behavior with mouseToggleDown() and mouseToggleUp().

2024.06.24 | AskUI SDK 0.20.0 and aiElement()

· One min read
Johannes Dienst
Developer Advocate

Introduction

This release includes two features we put a lot of effort into:

  • AI Element Creation Workflow (Windows only)
  • Convenience method expectAllExist which takes an array of descriptions for elements and returns if they are detected on the screen.

We also introduced a breaking change for the convenience method clickText() to make our API more consistent.


Breaking Changes

  • Method signature changed for clickText(). type becomes matching. API docs
// Before
await aui.clickText({text: 'Hello World', type: 'similar'});

// Now
await aui.clickText({text: 'Hello World', matching: 'similar'});

New Features

ADE commandlets to create and manage AI Elements:

Convenience method added to AskUI TypeScript SDK:


2024.06.17 | AskUI Suite 24.5.1 - Custom Element Enhancements

· 2 min read
Johannes Dienst
Developer Advocate

Introduction

We released the AskUI Installer version 24.5.1 which introduces new commandlets for error reporting and an overlay option for the AskUI Controller.

Also our Custom Element got a new implementation under the hood, so it works more consistently when changing the resolution of the screen. This change brings better stability to workflows that will be run on different machines. The update also expands the Custom Elements API of the AskUI SDK. You can now add a stopThreshold to optimize execution time.


Changes


Improvements

  • Custom Elements implementation is more robust against screen resolution changes

AskUI Installer Update 24.5.1

  • New commandlets to manage error reporting.
  • Status Indicator as an overlay of active screen.

Download link.


New Features

ADE commandlets:

AskUI Controller:

ADK:


Fixes

  • Drag & Drop (Mouse down, mouse move, mouse up) crashes controller

Versions

Components and Packages

  • AskUI Controller
    • AskUI Remote Device Controller: v0.3.0.0 Preview
    • AskUI Legacy UI Controller: 0.14.1
    • AskUI Remote Device Snipping Tool: v0.1.0.0 Preview
  • AskUI Runner: 0.1.4
  • AskUI Development Environment (ADE): Preview 3
  • AskUI Development Kit (ADK): 0.18.0

3rd Party Versions

  • Node.JS: 21.0.0
  • PowerShell 7.3.8

2024.04.29 | AskUI SDK 0.18.0

· 2 min read
Johannes Dienst
Developer Advocate

Introduction

We had a bunch of releases today which will help you be more effective and also to integrate AskUI better in your current toolchain.

Our docs lacked a comprehensive source of common challenges about detecting elements and how to address them. Thus we created a Recommended Practices page and a separate Custom Elements page. Let us know if something is missing!

We were also able to release more Convenience methods in this version. They are more expressive and also shorter to write. Read about them below and also how to migrate to the new syntax for clickText() and clickButton().

Speaking of convenience: We also added an ESLint rule that checks for the correct usage of expect().

Last but not least we fixed a bug in our test environment implementation @askui/jest-allure-circus. It is now compatible with jest-cucumber. (Link to accompanying blog).


Breaking Changes

  • Method signature changed for clickText(). API docs
// Before
await aui.clickText('Hello World');

// Now
await aui.clickText({text: 'Hello World', type: 'similar'});
  • Method signature changed for clickButton(). API docs

// Before
await aui.click().button().withText('Submit').exec();

// Now
await aui.clickButton({label: 'Submit'});

New Features

New Convenience methods added to AskUI TypeScript SDK:

We expanded our npm package @askui/eslint-plugin-askui with a new rule that checks for the correct usage of expect()


Fixes

  • @askui/jest-allure-circus accepts the testEnvironmentOptions addCodeInReport now
    • Upgrade the dependency to version 1.0.23
// Usage
const config: Config.InitialOptions = {
...
testEnvironment: '@askui/jest-allure-circus',
testEnvironmentOptions: {
addCodeInReport: false
},
};
...

2024.04.22 | AskUI Suite 24.4.1 - Inference Speedup

· 2 min read
Johannes Dienst
Developer Advocate

Introduction

We released the AskUI Installer version 24.4.1 which introduces new commandlets to start and stop the AskUI Controller, debug AskUI projects and processes.

Removed the icon classifier from the default inference. You can still target icons with icon().withText('<icon_name>') so the change is backwards compatible. But the name will not be displayed anymore. This discourages using the names of the icons as they are flaky across runs generally and often wrong.

Lastly we optimized our inference backend. This leads to faster workflow execution as it saves 300 - 700 milliseconds each time an instruction in an AskUI workflow invokes inference.


Changes

  • Removed the icon classifier from the default inference.

Improvements

  • Inference is faster 300-700 milliseconds

AskUI Installer Update 24.4.1

Introduces a number of new commandlets to manage your AskUI Installation: Download link.


New Features

This release focuses on new ADE commandlets.


Fixes

This release focuses on bug fixes.

  • High CPU usage of the RemoteDeviceController process, especially during idle.
  • All FFMPG processes are getting killed on AskUI Controller shutdown.
  • scroll() crashes AskUI Controller

Versions

Components and Packages

  • AskUI Controller
    • AskUI Controller: v0.2.1.0 Preview
    • AskUI Legacy UI Controller: 0.14.0
  • AskUI Runner: 0.1.4
  • AskUI Development Environment (ADE)
  • AskUI Development Kit (ADK)

3rd Party Versions

  • Node.JS: 21.0.0
  • PowerShell 7.3.8

2024.03.25 | AskUI SDK Automations Directory Option

· One min read
Johannes Dienst
Developer Advocate

Introduction

This week we introduced a new convenience method clickTextfieldNearestTo(label: string) which lets you click on a textfield with a label instead of a placeholder.

We also removed the convenience method dragTo as it was not working as intended.

There is a new option -ad --automations-description for initialising an AskUI project with npx askui@latest init. It lets you specify the name of the automations directory which was hardcoded to askui_example before.

Convenience Methods

We had to remove dragTo() as it was not working as expected


We introduced the following method:

clickTextfieldNearestTo(label: string)

Searches for an element of type textfield with a specific label nearest to it. If found, clicks it.

await aui.clickTextfieldNearestTo('E-Mail Address');

New Option -ad --automations-description for init

When you run npx askui@latest init you always got a directory askui_example. This broke backwards compatibility with existing installations of AskUI.

Thus we introduced a new option when running askui init:

-ad, --automations-directory <value>  a name for the directory askui stores its configuration and workflows in. (default: "askui_example")

You can use it like this to store the AskUI automations in a directory askui-Workflows:

npx askui@latest init -ad askui-Workflows

2024.03.04 | AskUI SDK Convenience Methods

· 2 min read
Johannes Dienst
Developer Advocate

Introduction

This week we improved on our docs. A guide for setting up your terminal and extensions in Visual Studio Code on Windows will improve your experience while developing AskUI Workflows.

We also fixed a display bug at the Supported Keys page.

In version 0.14.0 of the AskUI SDK we introduced some convenience methods to free users from writing repetitive code. This speeds up the development of AskUI Workflows and improves readability. Make sure to read the convenience docs which give you a detailed description on how to use each method.

Visual Studio Code Setup on Windows

Visual Studio Code is our recommended editor for developing AskUI workflows. Our docs for the AskUI Development Environment now have a section that helps you to setup Visual Studio Code for developing AskUI workflows efficiently:

  • Terminal configuration
  • Recommended extensions for Jest Runner, ESLint & Live View

Convenience Methods

We introduced the following methods:


clickButton(label: string)

Searches for an element of type button with a label and clicks it when found.

await aui.clickButton('Sign Up');

clickIcon(description: string)

Clicks an icon based on a description.

await aui.clickIcon('Logo looking like an apple');

clickText(text: string)

Searches for a text element and clicks it when found.

await aui.clickText('A big bear');

clickTexts(texts: string[])

Searches for text elements and clicks them one after another when found.

await aui.clickTexts(['A big bear', 'Brown Fox', 'Walks into a bar']);

clickTextfield(placeholder: string)

Searches for an element of type textfield with a specific placeholder text. If found, clicks it.

await aui.clickTextfield('E-Mail Address');

dragTo(element1: Executable, element2: Executable)

warning

This method is not working as intended and will be removed with the next release!

Drags element1 to element2.

Both must be a moveMouse() or moveMouseTo() instruction as in the example below.

await aui.dragTo(
aui.moveMouseTo().text('AskUI'),
aui.moveMouseTo().text('UI Automation')
);

pressKeyNTimes(key: PC_AND_MODIFIER_KEY, times = 2)

Press a key multiple times. At least two times.

await aui.pressKeyNTimes('right');
await aui.pressKeyNTimes('left', 3);

pressKeys(keys: PC_AND_MODIFIER_KEY[])

Press an array of keys one after another.

await aui.pressKeys(['right', 'left', 'enter']);

waitUntil(AskUICommand: Executable, maxTry = 5, waitTime = 2000)

Wait until an AskUICommand does not fail.

Use it to wait for an element to appear like this:

await aui.waitUntil(aui.expect().text('GitHub').exists());

2024.02.26 | AskUI Suite 24.2.2

· 2 min read
Johannes Dienst
Developer Advocate

Introduction

The main focus of the release AskUI Installer 24.2.2 was to polish the previous release 24.2.1 and therefore no major changes were introduced.

AskUI Installer Update 24.2.2

Introduces a number of new steps: Download link.

Installer for Windows

The installer contains a number of new and modified dialogues. Furthermore, it contains additional components.

More information can be found here.


New Features

This release focuses on bug fixes.


Improvements

This release focuses on bug fixes.

  • Improved the robustness of the inter-process connection between Legacy Controller and Remote Device Controller.
  • We only allow a single instance of the Legacy Controller for desktop scenarios (mobile platforms are untouched).

Fixes

This release focuses on bug fixes.

  • Fixed left and right arrow key mapping which were swapped.
  • Fixed issues with stuck mouse cursor and not finished actions.

Changes

  • Mouse movement uses the OS's cursor position set instead of delta movement on a 1000hz update rate. Making it more reliable.
  • Legacy Controller and Remote Device Controller are both shutdown with an error message if the connection is broken.

Known Issues

Mouse Movement

  • You may encounter issues when mouse pointer acceleration is enabled.
  • There might be issues with the mouse movement when it tries to move to invalid coordinates.

Versions

Components and Packages

  • AskUI Controller
    • AskUI Controller: v0.2.1.0 Preview
    • AskUI Legacy UI Controller: 0.14.0
  • AskUI Runner: 0.1.4
  • AskUI Development Environment (ADE)
  • AskUI Development Kit (ADK)

3rd Party Versions

  • Node.JS: 21.0.0
  • PowerShell 7.3.8

2024.02.07 | AskUI Suite & Installer

· 3 min read
Dominique Mader
Head of Software Engineering
Leon Meier
Head of Product

Introduction

AskUI offers a hassle-free installation experience for its products via a user-friendly installer. This installer facilitates the seamless installation of AskUI products with just a few clicks.

General

Installer for Windows

The installer contains a number of new and modified dialogues. Furthermore, it contains additional components.

More information can be found here.


New Features

We introduced a couple of new features.

In Preview


Known Issues

Installation Dependencies

  • AskUI Development Kit (former AskUI-Lib) is not included in the installer. It will be downloaded during the setup process.

Mouse Movement

  • You may encounter issues when mouse pointer acceleration is enabled.
  • There might be issues with the mouse movement when it tries to move to invalid coordinates.

Log Files

  • Log files disk space usage increased. They are stored in ~/.askui/ and can be deleted if that causes any issues.

Android

  • Recording on an Android device is not stable.
  • Recorded resolution might be lower than the device resolution.
  • Recorded video might be empty or single frame in cases no screen updates occurred during recording.
  • We don't support landscape mode on Android.
  • Devices with high DPI screen might not work as expected.
  • Video recording on MacOS is not supported.

Versions

Components and Packages

  • AskUI Controller
    • AskUI Controller: v0.2.0.0 Preview
    • AskUI Legacy UI Controller: 0.13.0
  • AskUI Runner: 0.1.4
  • AskUI Development Environment (ADE)
  • AskUI Development Kit (ADK)

3rd Party Versions

  • Node.JS: 21.0.0
  • PowerShell 7.3.8

To ensure a smooth onboarding experience, follow these steps:

  1. Download and Install:
    • Download and install the installer
    • Run the installer.
  2. Initialization:
    • Open the Command Prompt.
    • Type askui-shell.
  3. Configuration (First-Time Usage):
    • Configure AskUI settings:
      1. Type AskUI-SetSettings -WorkspaceId <askui_workspace_id> -Token <askui_token>.
      2. (Optional for Proxy Users) Type AskUI-SetSettings -HttpProxy <http_proxy_address> -HttpsProxy <https_proxy_address>.
  4. Verify Settings:
    • Type AskUI-ShowSettings to verify the configured settings.
  5. Create a New Project:
    • Create a new project by typing AskUI-NewProject.
  6. Start AskUI Controller:
    • Note: You can skip this if you selected Launch AskUI Controller at the installation wizard.
    • Start the AskUI Controller in background mode by typing AskUI-StartController -RunInBackground.
  7. Project Editing (Optional):
    • Open the project directory and starts editing the project.
info

You have to disable mouse acceleration for the AskUI Controller to work properly!

  1. Run the Project:
    • Run the project by typing AskUI-RunProject.

By following these steps, the user can efficiently set up AskUI Suite, configure essential settings, and seamlessly create and run projects.

2024.01.31 | OCR Model Enhancement

· One min read
Leon Meier
Head of Product

Introduction

We're excited to announce a speed enhancement in our OCR model, reducing processing times by 300 milliseconds.

Feature Update

OCR Model Speed Improvement

  • Faster text recognition, significantly improving workflow efficiency.
  • 1.62 sec down to 1.26 sec

Detailed Description

OCR Model Speed Enhancement

Our latest update has optimized the OCR model, achieving a notable speed increase while maintaining high accuracy.

Model: task='e2e_ocr' architecture='easy_ocr' version='1' interface='im2txtbox' use_case='faster' tags=[]
Average runtime: 1.2629041004180908
Standard deviation: 0.13027134152564313
Median: 1.2297358512878418

We are in the process of creating comprehensive documentation for our enhanced OCR model. We'd love to hear from you about what specific aspects or features you'd like to see covered in the documentation. Please share your thoughts and suggestions with us to ensure our documentation meets your needs and expectations.

Give Feedback >>>

2024.01.23

· 2 min read
Leon Meier
Head of Product

Introduction

We're introducing two significant enhancements to our platform: Single Sign-On (SSO) Integration for various identity providers and the AskUIXRayStepReporter for automation workflows in Xray for Jira. These features are designed to streamline the authentication process and improve test reporting, respectively.

Overview of Features

  1. Single Sign-On Integration

    • Documentation: SSO Setup Guide
    • This feature allows seamless integration with various identity providers including Azure AD, Google Workspace, and Okta, enhancing security and simplifying user management.
  2. AskUIXRayStepReporter

    • Documentation: AskUIXRayStepReporter Guide
    • This feature integrates with Xray for Jira to provide detailed reporting of automation workflows, including screenshots and test statuses.

Detailed Descriptions

Single Sign-On Integration

Our new SSO Integration feature supports a wide range of identity providers. It offers enhanced security, unified login experiences, and streamlined user account management, significantly reducing the overhead of managing separate login credentials.

AskUIXRayStepReporter

The AskUIXRayStepReporter is a powerful tool for reporting test automation results directly to Xray for Jira. It helps teams maintain comprehensive documentation of their test executions, complete with visual evidence and detailed status reports.


We are excited to see how these new features will enhance your experience with our platform, providing greater convenience, security, and efficiency in your automated testing and user management processes.

2023.12.13

· One min read
Leon Meier
Head of Product

Introduction

On December 13, we introduced three new features: the Visual Description Matching Method for selecting diverse visual elements, Targeted Annotations for precise debugging, and Precision Text Matching for flexible text matching.

Overview

  1. Visual Description Matching Method

    • Documentation: Matching Feature
    • This feature introduces the matching()method, allowing users to identify and interact with UI elements based solely on textual descriptions.
    • Enhances the flexibility of automated interactions with dynamic UIs.
  2. Targeted Annotations

    • Documentation: Annotate DetectedElements
    • Introduces an AnnotationRequest parameter in the annotate() method, allowing users to annotate specific elements retrieved from user interfaces by the get() method.
    • Simplifies debugging of UI element selection.
  3. Precision Text Matching

    • Documentation: Adjustable Similarity Score
    • Introduces an optional similarity_score parameter for the .withText() method.
    • Improves script resilience and minimizes mismatches in text-based automation.