Writing reflective unit tests to improve code quality

I’ve found that writing JUnit tests that do classpath scanning combined with reflection is a way to write unit tests that cross-cut the entire application. This can be useful to prevent anti-patterns, enforce code standards or just to prevent common dumb errors.

It also notable that tests like this will not just cover code that exists today, but code that get’s written in the future too.

Generally these tests won’t actually execute classes found the class path, but instead reflect on class structure, annotations or method IO. But you could also conceivably write tests that instantiated classes and tested behavior too.

Here’s an example of a classpath scanning, reflective test case.

Background: I’ve used warp-persist in the past to do JPA transactions with Guice projects. This gives you a @Transactional annotation which will apply a transaction across the method, unless exception(s) of specified types are thrown. Unfortunately, the warp-persist library has a design flaw, in that the default is that checked exceptions ROLLBACK, but runtime (non-checked) exceptions DO NOT. Why you’d want this is unclear to me, so we always end up writing {Exception.class, RuntimeException.class} to cover all cases – something that is easy to forget to do!

The test below, find’s all uses of the @Transactional annotation and verifies that each use of the annotation lists both RuntimeException.class and Exception.class as the rollback conditions.

By scanning the classpath, any new (or old) code will be flagged if the developer forgets this bit of housekeeping (this test has saved my bacon more than once).

package com.mycom;

import com.visural.common.ClassFinder;
import com.wideplay.warp.persist.Defaults.DefaultUnit;
import com.wideplay.warp.persist.Transactional;
import java.lang.reflect.Method;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.Set;
import junit.framework.TestCase;

public class TransactionalTest extends TestCase {

    public void testTransactionalDoesntCreateBadBugs() throws ClassNotFoundException, Exception {
        ClassFinder cf = new ClassFinder("com.mycom", true);
        List < String >  invalid = new ArrayList < String > ();
        Set < Class >  classes = cf.find();
        for (Class c : classes) {
            for (Method m : c.getDeclaredMethods()) {
                if (m.getAnnotation(Transactional.class) != null) {
                    List < Class < ? extends Exception >  >  ex = Arrays.asList(m.getAnnotation(Transactional.class).rollbackOn());
                    if (!ex.contains(Exception.class) || !ex.contains(RuntimeException.class)) {
                        invalid.add(c.getName()+"#"+m.getName()+"\n");
                    }
                }
            }
        }
        if (!invalid.isEmpty()) {
            Collections.sort(invalid);
            throw new Exception("The following methods use @Transactional, but do not rollback on Exception & RuntimeException: "+invalid.toString());
        }
    }
}

This is a slightly contrived example, in the sense that it is working around a design flaw in a 3rd party library. But, this technique is useful in many cases where you have design patterns, standards or anti-patterns which you want to enforce or prevent.

Here’s a few examples:

• Enforce all model classes should be Serializable
• Enforce class or method naming conventions
• Enforce specific package structures / hierarchies
• Ensure that arguments/returns are not overly specific e.g. return List not ArrayList

The tests can be as broad or narrow as your design rules. I’m not suggesting that the above examples are good ideas in all cases. But writing tests like these is a way to formalise a team/project standard in a more useful way, than writing it down in a document or wiki.

There be exceptions to many of the rules or patterns being tested, but you can always write the exceptions into the test.

Overall the advantages of this approach are:

• An opportunity to express the design pattern in the actual product code base as an executable test.
• A way to ensure that a new developer is made aware of the design pattern, at the time it is most relevant: when they need to apply it.
• A place to document the exceptions to the pattern, and why they are allowable.
• A way to keep everyone honest. Even experienced developers take short cuts, or are forgetful sometimes.

It’s a win all round. Not all Java developers are comfortable with class-path scanning and reflection/introspection (maybe they should be), but not all developers will need to write these tests, as they are probably best handled by a technical lead.

Footnote: if you want a general-purpose utility for doing class path scanning, then my library visural-common has ClassFinder.java – a way to search the class path for specific classes (as illustrated above).

Related posts:

1. NetBeans 6.7 Broke My Parameterized Tests
2. Testing Abstract Classes With Mockito and JUnit
3. Using Bottom-Up Iterative Object/Database Layer With hbm2java and warp-persist
4. ResourceTransformFilter – DataUris, LessCSS, Javascript and CSS Compression Made Easy!
5. HTML Sanitizer added to visural-common…