Category Archives: unit testing

My NetBeans Experience (part 1)

This will end up being a multi-part posting since I’m still evaluating NetBeans but my experience this morning has me really wondering if I can use it.  But not for the reasons you might expect.  NetBeans itself has been fine for me.  It’s a little weird to use their ant build if you have a build.xml as well but that’s another story.  Does anyone know if there is a way to show more than one row of files that you have open at the top?  It’s key bindings aren’t very friendly to Mac laptop keyboards also but those can be remapped pretty easily.

Anyways, on to important things.  I like to unit test my code.  I like unit tests, integration tests, etc.  And I like to use TestNG over JUnit4.  Why?  Because I can not stand the fact that JUnit4 requires your @BeforeClass and @AfterClass methods to be static.  There are other reasons as well but it just feels so awkward to have that requirement.  Anyway, the reason that impacts me with NetBeans is that there is not currently a TestNG plugin for NetBeans.  I want a way to easily start a test run with the debugger attached already.  Normally I don’t do this very often but this morning I needed to and I realized I can’t do that!  Of course you can find a way but that’s not really the point.  It’s a huge time savings to make this easy.  But I don’t like NetBeans more to justify the time actually build it myself.  Has anyone else out there started one?  I’ll help if you need some help, though.

The NetBeans editor is pretty good now.  I still think IDEA is better, though.  Part of that is just habit I think, though, since most of the features I use a lot in IDEA are now available in NetBeans.  So far it seems extremely stable in my use.  It’s easier and simpler to understand the file/project navigation than in Eclipse, also.  The built-in profiler has been awesome for me so far as well.  It has the font anti-aliasing now, too.  Well, maybe that’s just the Mac thing kicking in…not sure.

I do wish they did an auto-import like IDEA has.  Their import dialog can get a little overwhelming if you copy in a chunk of code and need a lot of imports resolved.  It’s easy to not realize you need to scroll down to verify more.  I think IDEA does this better going from one import to the next that it can’t automatically determine which instance of a type you are attempting to resolve .  IDEA’s ant editor seems better as well.

More to come another day.

Spring 2.5 Released

It looks like Spring 2.5 was released today. I’ve already been using the annotations for both service definition and web controller definition to greatly reduce the size of my configuration files. These are a great addition and I’m sure that I’ll be exploring them a lot in the future.

The other area that grabbed my interest was new integration test classes that support TestNG since previously they didn’t really seem to support TestNG with their test framework.

TestNG 5 is out

TestNG 5.0 is out. I’ll be upgrading to this one pretty much right away. I like the new tags like @BeforeClass and @BeforeMethod (and their corresponeding after annotations) which are a little more suitable than the @Configuration attribute in the previous releases. I had experimented with JUnit 4 a little bit and definitely liked the simpler annotations easier.

Could someone please make these default reports look nicer??? Yes yes, I know I could do it…I’m just hoping someone else would beat me to it.

The other annoying thing in Eclipse is that when I run a test by right clicking on it and running a TestNG test it creates a bunch of reports in the project directory by default. That’s not great default behavior. Why even create them by default when executing from within the Eclipse environment?

Mock Objects – How Useful?

I have a feeling the title of this post will probably have people thinking I’m going to bash on the concept of them and that’s not really the case but at the same time I think their usefulness tends to be limited to a relatively narrow set of cases.

Where I Have Used Them

Not in as many places as I thought I would. The main area I’m using mock objects (and by the way, I’m using EasyMock 2.0) is in my code that tests some filters that are based on Spring’s OncePerRequestFilter. It makes it easier to test the code when I mock the HttpServletRequest, HttpServletResponse, ServletContext and FilterConfig interfaces. I provide the data that the Spring filter requires and provides during its interaction with these interfaces based on the particular test details. For example, I’ll tell the ServletRequest mock to expect a call to getServletPath and then give it the response as well as setting the run once attribute setting. I’ll also set some items on the FilterConfig that the filter will need. The filter also needs to set an attribute on the request as well.

The reason that mocks come in handy here is that I don’t have to create dummy implementations of these larger interfaces just for testing purposes. Also, the interaction with these interfaces I mock is pretty simple. So basically, they’re great when you have code that interacts with large interfaces but where the interaction is generally limited. More complex interactions would make the mocks, and hence the tests, much more brittle.

Mocks in general work pretty well as long as you’re using interfaces. Easy mock has some features for mocking concrete classes but I haven’t used that feature so I can’t comment on it. They also have some strong limitations, IMHO, that I think should be aware of before going too far with them.

Why Don’t I Use Them More?

There are a number of reasons, but the first that always gets me is I don’t always know how code is going to call the interface if some of my code relies on code I didn’t write or maintain. It means I have a lot of iterations through test cycles before I know how to fully script the mock. Sure I could read through the code but that’s an even bigger pain than the approach I take (at least to me it is). Take my prior example for instance. I had to give the mock to the filter and see where it failed first. If the first call it makes is to the FilterConfig then that will fail. Then it gets past that and I see it fails when it asks for an attribute in the request. And this continues on until I get it right. But that part that gets me is that I’m trying to test my filter, which in this case is a redirect filter, but I’m really doing a lot of work just to get the code I rely to get to the point where I can test my redirect filter’s capabilities. What happens if they make subtle changes in their code but the external functionality no longer changes. My tests break without functionality actually breaking because the use of the interfaces that are mocked no longer respond as they should.

To be fair, if I had created dummy implementations of those interfaces they might not fare a whole lot better since methods I don’t think are going to be called will probably cause the test to fail later on as well. Maybe for common interfaces such as the ones I use above it would be handy to have an implementation that I could easily instantiate that is built for testing interaction with the interface. In general, using code that you didn’t write and then having to test your code on top of that code, will probably be a pain no matter what.

Another reason I don’t use mocks as much as I originally expected to is because in general most of my services are built with testability in mind so I don’t need to mock up a lot of interfaces to test them. I can feed them data and they just work with the test data. Mocks tend to work best in cases where I have to deal with external systems more than anything and there aren’t suitable classes available that take the necessary test data for the code to function the way I need it to for my tests. Overall, I’d really rather have my unit tests run through real code where possible vs. mocks because then I know the interaction works where as a mock could provide a disconnect with the real system in some cases.

I suppose I’m lucky that with my personal projects I have the luxury of building code for testability in the first place so I don’t have cases where I have no choice but to mock up a more complicated interface. In my work projects most of my code deals with code generation and persistence so mocks really aren’t all that necessary for my work. How are you using mocks? Are there scenarios I don’t have to deal with that you’ve come across and where they’ve really simplified your testing life?

Unit Test Code Coverage With Emma

Code Coverage Tools

While unit testing is a good start and it’s admirable to try and insure all your unit tests actually pass, it’s somewhat useless if you don’t know how much or what part of your code base doesn’t benefit from unit testing. Even the most committed developers of unit testing or TDD will miss covering some portion of their code. How do you try and prevent this from happening? Enter code coverage tools. The best known is probably Clover from Cenqua but for those of us working on our own home grown projects or that just like to use open source tools there is Emma.

Works Great!

I’ve been using it for MyThingo and it’s been very helpful. I’ve uncovered a few bugs that I didn’t know existed and hadn’t been uncovered by my unit testing up until now. Emma works by instrumenting your code and then you execute your unit tests using the instrumented class files. At runtime you set variables that tell Emma were to keep it’s coverage database and then after your tests execute you generate the report from that data. It seems to work very well and has a pretty comprehensive report of coverage including views of all your code with color coding to show which lines have and have not been executed during your tests.

Shortcomings

I have to admit that I miss the spit and polish that Clover seems to have. For example, Clover tells you how many times a line has been executed. While not critical it is a nice to have and I would expect this data is available in clover but just not exposed. If I get some time I may look into the report generating code and see what’s available. The integration of Clover in to Eclipse is also a very nice to have feature but again, isn’t critical.

Conclusion

Code coverage is one of those things that isn’t talked about much in unit testing circles. Okay, it is at times but not many developers I have worked with over the years really pay much attention to it. It is a critical part of your code testing infrastructure, however. You can’t just accept the fact that your unit tests pass. You must know how much of your code base is actually tested and then make sure that the parts that aren’t tested aren’t necessarily critical. It’s something I’ll harp on but in addition you have to make it brain dead simple. It has to be part of your build and not require any special attention by developers. It has to be part of your nightly build so reports are kept up to date as well. And finally, someone actually has to pay attention to the results and encourag developers to increase their coverage scores. If you do all this I think you’ll definitely see an improvement in the quality of the code that is delivered to QA and your users. And just think, better code going in to QA tends to mean shorter QA cycles which means faster delivery.

Using Hypersonic In-Memory for Unit Testing

Persistence Testing

One of the things that is always a pain when unit testing is when you need to test functionality that utilizes persistence. The unit tests that are easy are the ones that just test the functionality in your POJOs that doesn’t require persistence to test. Then there are the tests you want to write to make sure that your persistence configuration is correct. For example, you want to make sure that you’ve got your associations in Hibernate configured properly or that changes to data model don’t break things when you go to save. I’ve made mistakes before where I added a property that was required but some of my code didn’t set that property. It’d be nice to know that by having a unit test tell me that saving the object fails in certain situations.

The difficult part in this is getting a database configured. No one really wants to have one or more databases that are just for testing. It’s a pain to maintain and is going to be slow. When you unit test most times you are going to want to start with a clean slate and then populate with the data that is necessary for your tests. All of this overhead for a test? Not Ideal. What are some options?

Enter Hypersonic

Hypersonic is a Java database that can be run either on disk or in-memory. The in-memory is what we’re most interested in for testing because it doesn’t require a location on disk to run, is extremely fast since it doesn’t require disk access and is compact.

What I do is have a separate hibernate.cfg.xml file for unit testing that is identical in most regards to my production hibernate.cfg.xml except for the fact that it uses a straight JDBC connection vs. a DataSource reference for its connection source. Additionally, I tell Hibernate to create the database schema upon startup. Here is an example of some of the entries I set:

     <property name="hibernate.connection.url">
          jdbc:hsqldb:mem:testdb
     </property>
     <property name="hibernate.connection.driver_class>
          org.hsqldb.jdbcDriver
     </property>
     <property name="hibernate.dialect>
          org.hibernate.dialect.HSQLDialect
     </property>
     <property name="hibernate.connection.provider.class>
          org.hibernate.connection.DriverManagerConnectionProvider
     </property>
     <property name="hibernate.hbm2ddl.auto>
          create-drop
     </property>

The key here is the JDBC url, jdbc:hsqldb:mem:testdb, which tells Hypersonic to run in-memory vs. on disk and then the setting for hibernate.hbm2ddl which is create-drop and tells Hibernate to perform automatic schema generation.

I also use Spring as my primary application container/framework and so what I also do is have an applicationContext.env.xml for my normal application database environment and then a testApplicationContext.env.xml for testing. The main difference is the normal one configures the Hibernate SessionFactory with the production Hibernate configuration file and the test environment file configures it with the test Hibernate configuration. When I go to run unit tests that need both Spring and Hibernate I will use the test application context in place of the normal one. It’s pretty easy to do since I have a base class used for testing that will build out the application context during startup. It does this by just listing all of the Spring configuration files during startup and passing them in to the appropriate ApplicationContext sub-class.

Give this approach a try if you’ve been struggling with unit testing that involves database operations. It has saved me a lot of time and headache since I started using it. You can send me an e-mail if you have any questions about this.

Next up, code coverage with Emma.

Unit Testing with TestNG

A while back I started working on a project in the evenings that turned in to MyThingo. I decided from the get go to make sure I built exhaustive unit tests for the codebase. At the beginning I was using JUnit but I had read about TestNG a few times so I decided to give it a try. It was an amazingly fast transition. If you want to give it a try, just follow the documentation that comes with it.

After I started using it the little things that it does better are what kept me using it. Here are a few of the things that I use a lot:

Annotation Based Configuration

Rather than having to use a naming convention or extending a particular TestCase I can just mark a method as testable. I don’t need to have all my methods start with “test”. That gets annoying. Here’s an example of how it looks:

@Test
public void verifyAddition {
    assert 4 == Number.add(2,2) : "2 + 2 should have added up to 4";
}

Obviously the unit test above is not real but it shows how you mark a test method and that you don’t have to name it testVerifyAddition. The class this method is in is just a standard POJO as well.

Another annotation that comes in really handy is the ExpectedExceptions annotation. This tells TestNG that the method should fail unless it receives an exception of the specified type. Here’s the difference between the normal way of testing for a particular exception in JUnit and one in TestNG. First, the JUnit way:

public void testForNullArgumentException() {     
    try {         
        new Foo.doMethod(null);
        fail("I should have received a NullArgumentException");     
    } catch(NullArgumentException e) {         
        return;     
    } 
}

Now the way it’s done in TestNG:

@Test
@ExpectedExceptions(NullArgumentException.class)
public void nullArgInMethodX {
    new Foo().doMethod(null);
}

A little simpler, heh? None of what I’ve shown is impossible in JUnit (obviously) but overall it just makes testing easier. And the key to getting folks to unit test is to make it as painless as possible.

More Flexible Execution Profiles
One of the areas of JUnit that can drive a person crazy is that the class is dumped and loaded for every test. That means you have to setup and tear down after every test method. The idea is that you start with a clean slate for every test. But in the real world that’s not always ideal nor is it performant. Sometimes I just want to initialize items for the entire test class and other things for every method. TestNG lets you do this.

The way you do this is with the @Configuration annotation. This annotation can be added to methods and then its options allow to specify how the annotation affects the test class. For example, the beforeTestMethod parameter tells TestNG that the method should be run before each test method in the class. With beforeTestClass the method should be executed once at the beginning of the test run for this class. There are other options you can find in the TestNG documentation but these are the ones I use the most.

The last item I thought I’d go over is the ability to tell TestNG that a method X depends on the successful execution of other method(s) and if the dependent method(s) fails then there is no use in executing the test method in question. This can cut down on additional test execution times.

One of the issues I’ve had in getting other developers to unit test is that it’s usually just a pain to do tests beyond simple unit testing. You have to face facts that some developers just don’t care about it as much as you do. The key is making it as painless as possible and to make it relatively easy to mimic the environment that the code is going to run in. I’ve covered some of the areas where TestNG helps accomplish the simple/easier/faster part and in a future post I’ll cover some of the other areas such as setting up your Spring environment in conjunction with Hibernate using Hypersonic and auto-created database schemas using in-memory database. Another topic will be the use of easy mock for testing features that would normally require a container such as Tomcat. If you’re interested in those, leave me a comment or shoot me an e-mail and I’ll let you know when those are up and ready.