Hybrid monkey testing: Enhancing automated GUI tests with random test generation

Authors Thomas Wetzlmaier
Rudolf Ramler
Editors Tanja Vos
Sigrid Eldh
Wishnu Prasetya
Title Hybrid monkey testing: Enhancing automated GUI tests with random test generation
Booktitle Proceedings of the 8th ACM SIGSOFT International Workshop on Automated Software Testing (A-TEST'17)
Type in proceedings
Publisher ACM
ISBN 978-1-4503-5155-3
DOI 10.1145/3121245.3121247
Month September
Year 2017
Pages 5-10
SCCH ID# 17042
Abstract

Many software projects maintain automated GUI tests that are repeatedly executed for regression testing. Every test run executes exactly the same fixed sequence of steps confirming that the currently tested version shows precisely the same behavior as the last version. The confirmatory approach implemented by these tests limits their ability to find new defects. We therefore propose to combine existing automated regression tests with random test generation. Random test generation creates a rich variety of test steps that interact with the system under test in new, unexpected ways. Enhancing existing test cases with random test steps allows revealing new, hidden defects with little extra effort. In this paper we describe our implementation of a hybrid approach that enhances existing GUI test cases with additional, randomly generated interactions. We conducted an experiment using a mature, widely-used open source application. On average the added random interactions increased the number of visited application windows per test by 23.6% and code coverage by 12.9%. Running the enhanced tests revealed three new defects.