Minimizing Android GUI Test Suites
simple mobile app project ideas for students
In recent years, there has been a significant surge in the usage anddevelopment of apps for smartphones and tablets. Developers arewriting more apps for mobile platforms than for desktops. The com-plexity of mobile apps often lies in their graphical user interfaces(GUIs). Testing efforts of such apps mostly focus on the behaviorof graphical user interfaces.Several automated GUI testing techniques have recently beenproposed. The techniques include learning-based testing [8,29,31,32], model-based testing [1,23,45], genetic programming [27,28],fuzz testing [25,26,37], and static-analysis based approaches [4,33,34,44,49]. The goal of the majority of these techniques is to achievegood code and screen coverage (i.e. covering all distinct screens ofan app), and to find common bugs such as crashes and unrespon-siveness. Most of these techniques work by injecting sequences ofautomatically generated user inputs or actions to an app for severaltens of hours. We consider each sequence of actions injected bythese techniques to be atest case, and the set of all sequences ofactions to be atest suite.
Although automated GUI testing techniques could find bugs,they tend to generate large test suites containing thousands oftest cases. Each test case can contain tens to thousands of useractions. Such a large test suite can take several hours to execute,because the running time of a test suite is linear in the size of thetest suite.1However, regression tests should be fast so that theycan be used frequently during development. Therefore, such testsuites are difficult to use in regression testing.In this paper, we address the problem of generating a smallregression GUI test suite for an Android app. We assume that weare given a large test suite generated by an existing automatedGUI testing tool. simple mobile app project ideas for students We also assume that the test suite is replayablein the sense that if we rerun the test suite multiple times we getthe same coverage and observe the same sequence of app screens.(The evaluation section has details on how to obtain a replayable test suite from an automated GUI testing tool.) We assume thatthe test suite takes several hours to run on the app.
Our goal is tospend a reasonable amount of time, say a day, to generate a smallregression test suite for the app that runs for less than an hour andthat achieves similar code and screen coverage as the original testsuite provided as input.A couple of techniques have been proposed to minimize testsuites for GUIs. For example, Clapp et al. [7]and Hammoudi et al.[17]proposed delta-debugging [48] based algorithms. These tech-niques work well if the size of the input test suite is small, containingless than one thousand user inputs. However, they fail to scale forlarge test suites because they depend heavily on the rapid gen-eration and feasibility checking of new test cases. Unfortunately,for most real-world GUI apps, it takes few minutes to check thefeasibility of a new input sequence. Therefore, for large test suitescontaining tens of thousands of user actions, a delta-debuggingbased approach could take more than a month to effectively mini-mize a test suite. McMaster and Memon[30]proposed a GUI testsuite reduction technique for reducing the number of test cases ina test suite. However, this technique does not make any effort toreduce the size of each test case. In our experimental evaluation, weobserved that test cases generated by an automated tool can containsubsequences of redundant user actions, which can be removed toobtain smaller test suites.We propose an Android GUI test suite reduction algorithm thatcan scalably and effectively minimize large test suites. The keyinsight behind our technique is that if we can identify and removesome common forms of redundancies introduced by existing au-tomated GUI testing tools, then we can drastically lower the timerequired to minimize a test suite.
We manually analyzed test suitesgenerated by existing automated GUI testing tools and found thereare three kinds of redundancies that are common in these testsuites: 1) some test cases can be safely removed from a test suitewithout impacting code and screen coverage, 2) within a test case,certain loops can be eliminated without decreasing coverage, and3) many test cases share common subsequences of actions whoserepeated execution can be avoided by combining fragments fromdifferent action sequences. Based on these observations, we devel-oped an algorithm that removes these redundancies one-by-onewhile preserving the overall code and screen coverage of the testsuite.In order to identify redundant loops and common fragments oftest cases, we define a notion of state abstraction which enablesus to approximately determine if we are visiting the same abstractstate at least twice while executing a test case. If an abstract stateis visited twice during the execution, we have identified a loopwhich can potentially be removed. Similarly, if the executions oftwo test cases visit an identical subsequence of abstract states, weknow that fragments from the two test cases can be combined toobtain a longer test case which avoids re-executing the commonfragment. Whenever we get a new test case by removing a loopor by combining two fragments, the resulting test case may nottraverse the same abstract states as expected. In our algorithm,we check the feasibility of a newly created test case by executingit a few times and by checking if the execution visits the samesequence of abstract states every time—we call thisreplayability. https://codeshoppy.com/android-app-ideas-for-students-college-project.html We noticed that if our state abstraction is too coarse-grained ourfeasibility checks often fail, leading to longer running time.
On theother hand, if we use a too fine-grained state abstraction, we fail toidentify many redundancies. One contribution of this paper is todesign a good enough abstraction that works well in practice.One advantage of our algorithm over delta-debugging or otherblack-box algorithms is that we do not blindly generate all possiblenew test cases that can be constructed by dropping some actions.Rather, we use a suitable state abstraction to only drop potentiallyredundant loops. Another advantage is that we create new test casesby combining fragments from input test cases. This enables us tocome up with new, longer test cases which cannot be generatedusing delta-debugging or other test suite reduction techniques.Longer test cases are usually better than multiple shorter test casesbecause we do not have to perform a clean restart of an app. A cleanrestart of an app requires us to kill the app, erase app data, anderase SD card contents, which is very time consuming.
A longertest case in place of several shorter test cases avoids several suchexpensive restarts.We have implemented our algorithm in a prototype tool, calledDetReduce, for Android apps. The tool is publicly available at https://github.com/wtchoi/swifthand2. We applied DetReduce to severalapps and found that DetReduce could reduce a test-suite by a factorof 16.2×in size and a factor of 14.7×in running time on average. Wealso found that for a test suite generated by running SwiftHand [5]and a random testing algorithm [5] for 8 hours, DetReduce canreduce the test suite in an average of 14.6 hours. We are not awareof any existing technique that could get such huge reduction inthe size of a large GUI test suite in such a reasonable amount oftime. Note that DetReduce often runs longer than generating alltest cases; however, running DetReduce is a one-time cost. Once aregression suite has been generated, it will be run many times andeach run will take a fraction of the time required to generate alltest cases.