Automate your Java testing with Agitator 3.0

The hardest thing about testing is writing the tests in the first place; once you have tests, a whole panoply of tools can be applied to manage and report on those tests. But writing the tests, and writing them so they actually cover everything can be laborious and tedious, and tedium leads to errors. Enter Agitar's Agitator, a technique and a tool set, which sets out to ease test development and encourage test-first development.

Agitation, as a technique, is akin to handing your code to an assistant programmer who knows nothing of the context of the code and asking them to document everything that code actually does. They set about testing it by constructing code which feeds a range of possible values into each of the methods in the code and observing how variables change between the start and end of the method's code. They return to you with a list of these observations from which you select which ones you want to be tested for. In the future, when you change the code, you can ask the assistant to test the code according to those selected observations. Of course to do this, consistently, for every line of code is almost impossible and that's where Agitator comes in, automating the entire process and acting as a virtual assistant.

Agitator integrates into the Eclipse IDE as a plug-in or can be run from other IDEs, which launches Eclipse anyway. There are also command line tools for running on a build server to integrate with your development workflow. Within Eclipse, there's a new Agitation perspective which compresses the agitation process into pressing F9 to agitate a project. After agitation, the observations are displayed, ready for selection to become assertions.

The simplicity of code agitation disguises the complexity of scaling that technique up to typical codebases. Agitator manages to effectively hide that complexity with a simple interface and a lot of intelligence about how to handle generating tests without user intervention, without locking you out of being able to go in and change them. For example, creating that raw data to feed into the testing process is managed by factories; initially default factories are created so, for a native int, you get a factory which generates a random range of int values. Factories can, of course, generate class instances for testing too, including the automatic generation of mock objects based on existing types, using nested factories to generate parameter values for those mock objects. If you want to override the factory being used or test with a different generator, you can assign a new factory, varying the ranges or drawing them from an iterator or randomly from an array, generating random instances of a class. Specialised factories, for example generating servlet sessions, requests and responses, fill out the armoury.

Observations are expressed in Agitator's notation, which looks like a boolean expression, but includes functions like @PRE(x) which would be the value of 'x' before the test was executed and @RETURN which is the value the tested method returned. Agitator doesn't have to generate these observations; you can enter your own observation expressions, which is key to test first development. You can write a 'shell' method, manually enter the observations you want to be true and then agitate, at which point the assertions will fail and continue to do so till you write correct code to pass those tests. When an assertion fails, you can dig through to a snapshot of the test which reveals each iteration of the agitation where you can browse what values were fed in. Coverage is all important when testing and Agitator puts your coverage information right inside the editor, adding in the left hand side coverage tags which show how well each line is test covered, and flagging tested paths in green and untested paths in red. This is the signature of Agitator, putting testing and coverage results immediately in front of the developer, flagging untested or weakly tested code and reducing the effort in creating tests to remedy those omissions.

Agitation at its simplest; One view gives you code coverage and test results as you code.

Beyond the core agitation technique, Agitator builds out with Experts, which allow you to cut across a project with a set of default rules of your own creation. Say you have a convention that all methods that begin with "do" should return an integer within a range, you can add an expert which matches the method name using a regular expression, and then create an assertion to enforce that rule. Whenever Agitator matches a rule, it flags it up in the Experts panel and editor and asks if you want to apply the associated observations. Agitator comes with some ready constructed Experts for J2EE and Struts, which is useful, but the feature doesn't feel as tightly integrated as the core functionality and the Experts UI feels laborious. That said, Experts has a lot of potential for embedding cross team coding discipline.

Agitator is a powerful tool, pulling together the development and reporting of tests deep into the coding cycle. What Agitator isn't is a magic bullet; throw it at an existing legacy project and you still, quite reasonably, have to figure out which observations should become assertions, and even then you have only the skeleton of default tests. It is best introduced to a legacy project in an organic fashion where you can focus on getting high coverage in small areas. For new projects, Agitator's positive encouragement of test first development could well make it an essential tool. The only possible inhibitor is the cost which starts at AU$590.00 per developer per month including support for the subscription model, or AU$5,600 per seat plus support for 10 licences to purchase outright. You really need to be sold on agitation, and there's no "Agitator-Lite" to get you hooked, but trust us, once you've agitated your code, you won't want to go back to hand crafting all your testing.