Where do you put your Tests?
October 17, 2006, 11:15:13 pm

There are two schools of thinking when it comes to unit tests and what package to put them in. Generally speaking, a test class usually exists one to one with an actual implementation class. This is not a given, but it is a common pattern you see.

In the VisualAge world, when we made tests, we would put those tests in to a separate application from the application that contained the real code, eg:

MyRealProgramApp and MyRealProgramTestApp

The reason for this was simple - separating out the tests at 'package' time is difficult and annoying. Especially when you create 'Release' configuration maps.

So when Steve Aldred and I moved over to VisualWorks in our spare time, we copied across the same pattern. We'd put our tests in to a different package to the package that contained the code. This allowed us to simply not load those packages when we wanted to package.

But things for us have changed since then. We're now distributing production code /with/ its test cases - ie: we don't strip out the test cases. Why? - So that we can verify that in the production environment the tests still behave as expected. This is a huge win.

As well as that, we learnt a lot more about the packager in VisualWorks - more specifically, we learnt how little it does for you when you want to strip things. So, we use scripts to drive the packager and our scripts can do things like filter tests out of the image regardless of what package they're in.

So that means that our assumption that it's difficult to remove tests from a package is now no longer true. And our assumption that you may not want to deliver the tests in a production piece of code are also no longer true.

On top of this, that charismatic person called Travis Griggs opened my eyes to a different pattern - leave the tests in the package where the real code is. I initially found this to be annoying. It cluttered up the package in ways I wasn't expected. I was taken aback, I didn't know what to do.. I was confronted with something that I thought was wrong.

But after breaking away my assumptions, I started to appreciate Travis's pattern more. It means that the tests are always -right there- where the code I'm writing is. The built in testing tool is easier to use when the tests are -right there- too - you don't have to go looking for them after you've changed some code.

So.. I now put the tests in to package that contains the code.

What do you do?

By on October 18, 2006, 6:29:39 am

I disagree with "It means that the tests are always -right there- where the code I'm writing is". If you're working on Class X, then the tests are not "there" but in Class TestCaseX and maybe in TestCaseY and TestCaseZ.

Putting the tests in seperate packages also has the advantage that you can deploy an image without the tests, and then load the tests via parcels into the deployed image to run the test there.

And in Panda's/Pollock's case, I really don't want to always have 28000 tests in my image or deployment.

By Travis Griggs on October 20, 2006, 3:08:12 am
By Runar Jordahl on October 20, 2006, 3:36:22 am

I organize tests in separate packages (Store) or applications (Envy).As pointed out, structuring code like this can help during packaging, but I do not think this is the most important aspect. More important is that using packages communicates the design ideas of the code. As an example, putting a method in the “Test”-package, clearly communicates that the method is not to be used from the “regular” implementation code. Along the same lines I would also separate UI and domain code. If the UI needs to add extra methods to the domain, (class) extensions can be used.While many consider structuring the code (of a single program) into packages/applications an academic exercise, I believe separating the various aspects of your code into components and defining relations between the components (prerequites) can ease the understanding of a complex application. 

By on October 26, 2006, 4:43:24 pm

I put my tests in packages.  The single main reason for this is that I use method wrappers a lot in tests.  Typically, my model-layer tests drive values into the model-layer code at as high a level as possible.  Then they extract values, also at as high a level as possible, and verify that they were the values wanted (or, too often, something completely different :-)

Often enough, the UI layer is doing exactly the same thing - driving values into the top-level of the model layer and getting results back.  Thus I aim to have lots of UI testing but to write very few UI tests as such.  Instead my UI test classes reuse my model-layer test classes, adding the test's start values via the widgets instead as top-level interface parameters, invoking buttons or whatever instead of calling interface functions, and checking the results via other widgets.

Where do the method wrappers come in?  Well, ideally widgets would always be findable, and launching code would always return what it launched instead of self (hey, I sent the message to the receiver so it's the one object I definitely do not need returned! :-/).  In the real world, if you want to call from as close to the user as possible and you want robust UI tests in the face of changing UIs, forked popUps and so on, then a method wrapper or so often makes it doable.  They can also save you from the effort of creating mock objects in many situations.

So far, so good.  Problem, not everyone wants method wrappers in their delivered image.  I can understand that.  So I have the model-layer and UI-layer tests in separate packages, applications or whatever in part to hold the method wrapper prerequisites separate.

As I use RB buffers / views a lot, I'm rarely concerned about things appearing together in whatever default CM layout I'm looking at.  The first buffer has my production code, the second my tests, and there will usually be several more before I'm done;     swapping between them is a keyboard shortcut.

By Niall Ross on October 26, 2006, 4:59:09 pm

The above comment was submitted by Niall Ross.