Unit Tests Don’t Bite

You might have heard about the term “Test Driven Development”. The idea is – as I understand it – that for every problem you find in a component of your apps you create a Unit Test that fails. Then you fix the bug. The Test now passes. This can be carried further by writing your test cases even before you write any implementation code.

Especially when encapsulating your frequently-used code in static libraries or frameworks those unit tests can help save you a lot of grief. Imagine adding some nifty new feature to inadvertenly  introducing a bug that would break some other existing functionality. If you run the unit tests they would show you immediately that your change broke something.

In this blog post I’m summarizing a couple of things to help you get your own unit tests started.

Unit Tests are set up in Xcode like any other target. They have a product with extension octest which contains the code and resources for the test. So just like with any other target you have to assign the source files and resources to the unit test target.

For this demonstration I added several tests for DTHTMLParser to my DTFoundation project which you can get on GitHub.

Step 1: Setting up a Test Bundle

You start up by adding a new “Cocoa Touch Unit Testing Bundle”.

This adds this as a target to your project, creates a subfolder by the same name below your project root and places an empty test case class there as well. The bundle will also contain an Info.plist and a precompiled header file (PCH).

I generally don’t like this as I have a certain project structure I want to enforce. This has a “Test” subfolder below the project root, there you find the PCH and Info.plist. In the “Source” subfolder I want to keep the header and implementation files. In the “Resources” subfolder I keep all other resources, for example HTML files for testing. The structure looks like this:

You can choose your own file system structure if this one doesn’t fit your needs. You just have to make sure of the following items:

• All resource files need to be assigned to the test target
• All implementation files (.m) need to be part of the target
• No header files (.h) are part of the target
• Neither are Info.plist nor PCH

If you change the structure like you want you can remove the references to the previous items by reference and then add the new structure. If you do you can choose to add all to the files and then afterwards you manually remove the plist and PCH from the target.

As a reminder you can modify target membership of files either in the right sidebar or in the build phases, resources are in “Copy Bundle Resources”, source files are in “Compile Sources”.

The plist and PCH are referenced not by being a member of the target, but by path in the build settings. As mentioned above the Xcode template for test bundles puts them into a subfolder, you you might need to adjust their path as well.

Here you can see that I like to have the product name short and without Spaces, while the target name is longer and more descriptive. The default setting for a new target to use the target name as product name. You are free to adjust that in the build settings as well.

Step 2: Adding Code Tests

I recommend to put all tests related one high level class into a HighLevelClassTest class. For this purpose you add that to your Source folder. “New File…” – “Objective-C test case class”.

The individual test cases are methods that are prefixed “test”. Let’s add a very simple test case to make sure that the initializer returns nil when passed invalid parameters.

// DTHTMLParserTest.h
@interface DTHTMLParserTest : SenTestCase
- (void)testNilData;
@end
 
// DTHTMLParserTest.m
#import "DTHTMLParser.h"
 
@interface DTHTMLParserTest ()
 
@end
 
@implementation DTHTMLParserTest
- (void)testNilData
{
	// try to create a parser with nil data
	DTHTMLParser *parser = [[DTHTMLParser alloc] initWithData:nil encoding:NSUTF8StringEncoding];
 
	// make sure that this is nil
	STAssertNil(parser, @"Parser Object should be nil");
}

Of course you need to add an #import for the class header you are testing. Then you do the operations you want to test. After each individual test step you can use the STAssert macros. Those are behaving similar to NSAsserts, if the first parameter is not true then the test is aborted and the description in the second parameter is being logged.

The most common STAssert macros you will use are:

• STFail – fails the test right away
• STAssertNil – parameter must be nil
• STAssertNotNil – parameter must be not nil
• STAssertTrue – parameter must be true, YES, non-0
• STAssertFalse – parameter must be false, NO, 0
• STAssertEquals – Generates a failure when a1 is not equal to a2. This test is for C scalars, structs and unions.
• STAssertEqualObjects – the first two parameters are object that must be equal to pass (isEqual:)

Executing test code is just the same as executing app code. So you can even debug your tests by adding breakpoints to pause execution for inspecting variables.

Step 3: Using Bundle Resources

In iOS apps you would usually get the path to bundle resources via [[NSBundle mainBundle] pathForResource:ofType:]. This does not work for unit tests because there is no main bundle. Rather you have to first get the NSBundle that contains the test class and from there you can get the path for the resource.

We have a processing_instruction.html file as member of the test target and a test using this would look like so:

- (void)testProcessingInstruction
{
    NSBundle *bundle = [NSBundle bundleForClass:[self class]];
    NSString *path = [bundle pathForResource:@"processing_instruction" ofType:@"html"];
    NSData *data = [[NSData alloc] initWithContentsOfFile:path];
    DTHTMLParser *parser = [[DTHTMLParser alloc] initWithData:data encoding:NSUTF8StringEncoding];
    parser.delegate = self;
    [parser parse];
 
    STAssertTrue([parser parse], @"Cannnot parse");
    STAssertNil(parser.parserError, @"There should be no error");
}

The basic process is that you set up a test, perform a step, assert some result, perform another step, assert again and so on. Any test that reaches its end without an assert failing it is considered passed.

Step 4: Configuring a Scheme for Testing

As of Xcode 4 we got several things you can do for any target: Run, Test, Profile, Analyze. Each of these can be configured separately. The kind of action you want to perform on the currently selected target can be chosen by long pressing on the round button in the upper left hand corner.

Xcode creates a scheme for each new target by default. These schemes (unless shared) are only visible for you. Another user checking out the project will again have schemes for all targets auto-generated. As far as I know you cannot prevent this process if the other user has “Autocreate schemes” checked.

A scheme for a Unit Test target is quite unnecessary, so we can safely remove that. Instead we will tell Xcode that these tests belong to the “Static Library” target.

Usually you would set up your unit tests to be the Test option for an app. Here in DTFoundation there is no app, so we configure the unit tests for the static library target.

Product – Edit Scheme. Then add the Unit Test target via the Plus button and optionally uncheck individual tests you don’t want to be performed.

Having done that you can perform a one-off run of the tests. Long-Press on the top left round button and choose test. This will perform the tests for the selected scheme, “Static Library” in this case. Usually you would run unit tests on the simulator.

After the test was run you can check the log to see if all tests worked. For demonstration purposes I added a test that always fails with STFail. You will only see lines for the test cases plus what is logged from STAssert macros. If you have an NSLog in the test you can see that if you open up the details panel.

The green checkmarks are just as soothing as the red exclamation marks are annoying. That in itself should motivate you to get all tests to be green as soon as possible.

Step 5: Testing with Every Build

With the setup from the previous test you can manually run the tests via the round button or CMD+U. I sometimes makes sense to force yourself to test with every build of a target because you might start getting lazy always having to push CMD+B for building and CMD+U for testing.

You can easily automate the process to have the tests automatically occur whenever you execute a build so that you immediately know if you have broken something.

In the Build Settings of the unit test target set the “Test After Build” flag to YES. This causes the tests to be run whenever the unit test bundle was built.

Then you edit your scheme and add a checkmark in the Run Column of the Unit Tests.

With this change you will see that the Unit Tests are built and executed right after the building of the main target.

If you have a sufficiently fast Mac then you don’t notice the processing time needed for the tests at all. If you have a larger project you might want to refrain from testing with each build but rather have your continuous integration server (e.g. Jenkins) perform the Unit Tests on each check in.

Step 6: Using Specific App States and Locations

Let me mention for sake of completeness that there are two interesting tools for use with unit testing. Look at the graphic in Step 4 and you see two columns which you can individually set for each test.

Those allow you to have a specific app state being restored from a package before the testing commences and a given location/path. The application data bundle you can retrieve from a device running the app via Organizer. The location data can either be a location from a list or you can even add a GPX file to simulate movement along a path.

Though I have to admit never having tried these out myself. I’d be interested to hear about your experiences with these.

Conclusion

Unit Tests are easy to create and can save you from some unpleasant surprises when somebody (or even you) breaks something that was working previously. Even if you don’t subscribe to the “Test-driven Development” paradigm it is still something worthwhile because even just the act of thinking what could go wrong with your classes makes you code more defensively.

Ultimately your software will be less prone to accidentally re-introduced bugs – provided that you add extensive tests for each bug you fixed.

Categories: Recipes