Ranorex can run UI automation tests at a very fast pace. But as test suites and repositories grow, they become more complex and execution speed tends to suffer. Christian Breitwieser, one of Ranorex’s excellent test automation engineers, and I ran some experiments to get a better handle on some of the factors that can affect execution speed.
I plan to write a series of posts on ways to improve the test run speed. I’ll link to them from this post as they go live. The unlinked posts are tentative.
In general, a number of factors affect the performance of the test run. They generally fall into the following groups:
- Environmental – includes screen resolution; memory and CPU; number of running foreground processes; etc.
- Tooling – includes whether the debugger is attached; whether the tests are run from the command line, Visual Studio or Ranorex Studio; etc.
- Ranorex behavior – includes the repository cache; the flexibility of the RXPaths; etc.
Some of these factors have a more evident effect and some less obvious. I’ll try to tackle some of these factors in the series.
The experiments all use the KeePassTestSuite sample project (in C#) that comes with Ranorex Studio, so you can try to reproduce them on your own. I used Ranorex 4.1 to run all of these experiments on my system, which has the following characteristics:
- Windows 8.1 running on a Boot Camp partition in Parallels 9
- 8 GB RAM and 4 logical CPUs assigned to the Windows VM
- SSD drive
Your system and the resulting numbers will probably be different, but the conclusions should be similar. For the most part, these conclusions bore out on most of the systems that I’ve worked with.
Let me know in the comments if you have any specific test run performance questions. I’ll try to address what I can in the upcoming posts.