Accelerate Test Execution
To speed up the CI process, we developed a group of features that support every phаse of the process, from test creation to execution. Let’s take a look at how BELLATRIX can speed up your test execution by distributing your tests on multiple machines, enhanced finding and waiting of elements, and providing an improved app life cycle control.
Parallel and Distributed Test Execution
Test execution runner that speeds up your test execution using your whole hardware capacity. BELLATRIX comes with its test runner- Meissa, that can run tests on Windows, MacOS, and Linux. Meissa speeds up the tests by distributing them on multiple machines. The test execution can be further improved by executing the tests in parallel through tests agents.
Smart Tests Balancing
Smartly balances the tests across remote machines based on previous tests execution times which can significantly improve the overall test run time.
Failing Tests Retry
Has the capability to retry failing tests multiple times to see whether there are bugs in the tests, real problems in the system under the test, or some random environmental issues occurring during the run.
Safe Test Run Abortion
Keeps the clean state of test agents and provides a fast test run abortion.
Executes custom logic through plug-in API without modifying the source code.
Optimized Element Finding and Waiting
Better finding and waiting of elements out of the box and flexible setup.
One of the most significant problems in test automation is handling timeouts and performing actions on elements which may not be on the screen right now.
If you use global timeouts such as WebDriver implicit wait.
AndroidDriver<AppiumWebElement> driver = new AndroidDriver<AppiumWebElement>(new Uri("http://127.0.0.1:4723/wd/hub", desiredCaps)); driver.Manage().Timeouts().ImplicitWait = TimeSpan.FromSeconds(30);
In some cases, you may need a larger wait interval. What do you do if this happens? One option is to increase the global timeout, but this will affect all existing tests, slowing down the whole test run significantly.
Another option is to use the global timeout and occasionally add hard-coded pauses in your tests where needed.
However, this is the worst thing you can do. One time you may need to wait 1 second, and another time 5 seconds. If you always use the biggest possible pause, your tests will get slower and slower.
BELLATRIX hides the whole complexity of searching and waiting for elements. All elements are returned immediately when they are present on the page in the right state.
Optimized App Life Cycle Control
Reuse the app and configure it when to restart.
Two of the operations that take the most time in Android tests are related to starting and closing the application under test. Each of these operations can take from a fraction of a second to several seconds. If you open and close the app for each test and you have thousands of tests, this can make your test runs more than twice as long!
To battle this problem, BELLATRIX offers an automatic way of handling the app life cycle across multiple tests. You need to set this behavior in the Android attribute applied to each test class.
[Android(Constants.AndroidNativeAppPath, Constants.AndroidDefaultAndroidVersion, Constants.AndroidDefaultDeviceName, Constants.AndroidNativeAppAppExamplePackage, ".view.Controls1", AppBehavior.RestartEveryTime)]
The app life cycle is configured through the AppBehavior property. Available options:
For each test a separate Appium instance is created and the previous app is closed.
The app is only restarted if the previous test failed.
The app is only restarted if the previous test’s app was different. In all other cases, the app is reused if possible.
If you use one of the “restart on fail” or “reuse if started” options, it can significantly decrease the test execution time, eliminating the additional time for browser initialization and disposal.