TestEvoHound

TestEvoHound is a static analysis tool, which allows to analyse multiple revisions of a software project for test fixture smells.

Overview of TestEvoHound

We developed TestEvoHound to analyze fixture smell evolution. TestEvoHound works with Git and SVN repositories. When analyzing code, TestEvoHound executes four tasks.

During the Revision Checkout task, TestEvoHound checks out each revision of the project under analysis, and for each revision it starts the Build Process task. Here, the tool searches for ANT or MAVEN build files, initiates the build process and compiles the source code (including tests). Then, the Test Fixture Smell Analysis task invokes the TestHound tool to analyze the current revision for smells and stores the outcome. Finally, when all revisions have been analyzed, the Trend Analysis occurs.

Here, the TestEvoHound tool calculates the trends and measurements among all revisions. This information is stored in comma-separated value format to allow easy visualization by tools like Excel or R.

What TestEvoHound solves

An important challenge in creating automated tests is how to design test fixtures, i.e., the setup code that initializes the system under test before actual automated testing can start. Test designers have to choose between different approaches for the setup, trading off maintenance overhead with slow test execution. Over time, test code quality can erode and test smells can develop, such as the occurrence of overly general fixtures, obscure in-line code and dead fields. In this paper, we investigate how fixture-related test smells evolve over time by analyzing several thousand revisions of five open source systems. Our findings indicate that setup management strategies strongly influence the types of test fixture smells that emerge in code, and that several types of fixture smells often emerge at the same time. Based on this information, we recommend important guidelines for setup strategies, and suggest how tool support can be improved to help in both avoiding the emergence of such smells as well as how to refactor code when test smells do appear. An important challenge in creating automated tests is how to design test fixtures, i.e., the setup code that initializes the system under test before actual automated testing can start. Test designers have to choose between different approaches for the setup, trading off maintenance overhead with slow test execution. Over time, test code quality can erode and test smells can develop, such as the occurrence of overly general fixtures, obscure in-line code and dead fields. In this paper, we investigate how fixture-related test smells evolve over time by analyzing several thousand revisions of five open source systems. Our findings indicate that setup management strategies strongly influence the types of test fixture smells that emerge in code, and that several types of fixture smells often emerge at the same time. Based on this information, we recommend important guidelines for setup strategies, and suggest how tool support can be improved to help in both avoiding the emergence of such smells as well as how to refactor code when test smells do appear.

Study details

For the statistic correlations we looked at the following characteristics to determine the uniqueness of an event:

  • the name of the class;
  • the test methods of a class;
  • whether it has a setup, class setup, tear down or class tear down;
  • presence of the vague header smell;
  • presence and degree of the general fixture smell; name of methods affected;
  • presence and degree of the Test Maverick smell; name of methods affected;
  • presence and degree of the obscure In-line setup smell; name of methods affected;
  • presence and degree of the LCOTM smell;
  • presence and degree of the dead field smell;
  • presence and degree of the dead inherited field smell;
  • which fields are in the class; name and type of the fields;
  • how many setup instructions are contained in the setup procedure;
  • how many setup instructions are contained in the class setup;
  • what is the super class;

Download

We are currently in the process to make TestEvoHound available for download. Please revisit in a few days or feel free to contact me via email.

-- MichaelaGreiler - 05 Feb 2013

Revision: r1.4 - 20 Feb 2013 - 10:08 - MichaelaGreiler
MichaelaGreiler > TestEvoHound
Copyright © 2003-2017, Software Engineering Research Group, Delft University of Technology, The Netherlands