Localization Unit Test

It happens to the best: you add new features to an app localized in several languages. You are not the lazy type that names the NSLocalizedString key’s the same as the English-language strings. Instead you name the keys semantically, like DOWNLOAD_ALERT_MSG.

Then when it comes to shipping you send the client a note that there are some new strings and assume that he will be able to find out which are new and need to be translated. Which he would if he was using a tool like Linguan. Which he is not, because you might have forgotten to recommend it to him, or if he’s extremely unlucky then he’s a Windows user.

Long story short, the app ships and then you are beginning to receive LOL-adorned tweets with screen shots attached that show the placeholder string.

Note: this example is totally constructed, it didn’t happen in real life, and certainly not to us. But maybe it happened to you, maybe in a milder form if you have less obvious placeholder keys. In that case you might have gotten reports from German users complaining that certain texts are stubbornly English.

Creating a Localization Completeness Unit Test

Since we never (again) want to come into such an awkward situation we decided to treat app builds which are missing localizations as defective. By our definition if an app is missing tokens it is just as bad as if it where crashing on launch.

This is exactly the kind of thing that prompts you to add a unit test for. The question that the unit test should be asking is: Is there a localization for every string macro? For every language?

In order to be able to ask and answer these questions we need a few ingredients. We need to be able to scan our source code for occurrences of NSLocalizedString and friends. (Or our own custom prefix if we have one) And we need to get the strings files loaded so that we compare these against the list of all tokens.

For this purpose I developed DTLocalizableStringScanner. This is basically my Open Source alternative to genstrings, several orders of magnitude faster and you can link it into your apps or unit tests as a static library. DTLocalizableStringScanner scans source code files for occurrences of the localization macros and aggregates the results into one or more string tables.

One recent addition to this component is a parser for strings files which we developed as the counterpart. By the way: DTLocalizableStringScanner is also part of the above mentioned Linguan app.

Setting Up

For my demonstration we create a simple iOS app. Add a Localizable.string files and set that to be localized in German and English. The demo project is available on our Examples repository.

How-To Reminder: Add new Resource, Localizable.strings. Click on “Localize…” button in inspector. Set the project localization languages in the project settings.

To have something to test against, I added these macros to ViewController.m:

NSLocalizedString(@"ALERT_VIEW_MSG", @"Message in an alert view");
NSLocalizedString(@"ALERT_VIEW_TITLE", @"Title in an alert view");

Let’s assume that the title is a part of a new feature and we only have the ALERT_VIEW_MSG already localized. So we add the key value pair into both Localizable.strings files.

This is the starting point for our experiment. The unit test should fail and show us that ALERT_VIEW_TITLE is missing a localization in both English and German.

The unit test itself is a new LocalizationUnitTest Mac target. I prefer Mac for unit tests that don’t have iOS-specific code because those don’t need to launch the iOS simulator.

Getting the Project Folder Path

Step one for our unit test is that we have to scan the current version of the source code for string macros. The tricky part here is that you need to find out the project root path. One way how to get that I have seen was to add a compiler option that converts the $PROJECT_DIR environment variable into a define which is then available at run time.

My own approach is slightly different. I use the __FILE__ predefined macro to get the source code path of the current file and from there I search for the Xcode project file. This still requires that you set the name of the project file but I don’t think that this changes that often.

/**
 Determines the main project path during runtime
 */
- (NSString *)_projectPath
{
    NSString *pathOfFolderOfCurrentSourceFile = [[NSString stringWithCString:__FILE__
             encoding:NSUTF8StringEncoding] stringByDeletingLastPathComponent];
 
    NSString *currentPath = [pathOfFolderOfCurrentSourceFile copy];
 
    BOOL foundIt = NO;
 
    do
    {
        NSString *testPath = [currentPath stringByAppendingPathComponent:@"LocalizationDemo.xcodeproj"];
 
        if ([[NSFileManager defaultManager] fileExistsAtPath:testPath])
        {
            // found it
            foundIt = YES;
            break;
        }
 
        if ([currentPath isEqualToString:@"/"])
        {
            // cannot go further up
            break;
        }
 
        currentPath = [currentPath stringByDeletingLastPathComponent];
    } while ([currentPath length]);
 
    if (!foundIt)
    {
        return nil;
    }
 
    return currentPath;
}

The second step is to find the source code files to scan. In a LocalizationUnitTest class I added a method to find the paths of the source code files.

/**
 Creates a list of all source file URLs at this project path
 */
- (NSArray *)_sourceFilesAtProjectPath:(NSString *)projectPath
{
 
    NSMutableArray *tmpArray = [NSMutableArray array];
 
    NSArray *keys = @[NSURLIsDirectoryKey];
 
    NSDirectoryEnumerator *enumerator = [[NSFileManager defaultManager]
                                         enumeratorAtURL:[NSURL fileURLWithPath:projectPath]
                                         includingPropertiesForKeys:keys
                                         options:(NSDirectoryEnumerationSkipsPackageDescendants
                                                  | NSDirectoryEnumerationSkipsHiddenFiles)
                                         errorHandler:^(NSURL *url, NSError *error) {
                                             STFail(@"%@", [error localizedDescription]);
                                             // Handle the error.
                                             // Return YES if the enumeration should continue
                                             return NO;
                                         }];
 
    // enumerator goes into depth too!
    for (NSURL *url in enumerator)
    {
        NSNumber *isDirectory = nil;
        [url getResourceValue:&isDirectory forKey:NSURLIsDirectoryKey error:NULL];
 
        if (![isDirectory boolValue])
        {
            if ([[[url path] pathExtension] isEqualToString:@"m"])
            {
                [tmpArray addObject:[url path]];
            }
        }
    }
 
    return [tmpArray copy];
}
 
/**
 Builds up the string table of entries that require localization
 */
- (void)_scanSourceCodeFiles:(NSArray *)sourceCodePaths
{
    // do work
}
 
- (void)setUp
{
    [super setUp];
 
    // get the project folder
    NSString *projectFolder = [self _projectPath];
 
    // gets file URLs for all the 
    NSArray *sourceFiles = [self _sourceFilesAtProjectPath:projectFolder];
 
    // scans the source files and adds the resulting to the string table
    [self _scanSourceCodeFiles:sourceFiles];
}

At this point we get a list of all the .m files contained in our project. For the actual macro scanning functionality we now need DTLocalizableStringScanner. In my example I added it as git submodule and adjusted the header search path in the LocalizationUnitTest build settings accordingly.

I assume at this point that you know how to add a git submodule and link a target from it into your own project. If not, then please look at this example’s project.

Scanning the source code files for string macros is quite straightforward. For sake of simplicity I’m only working with the single default Localizable.strings table and I’m not using any custom parameters available on this component, like for setting a custom macro prefix.

/**
 Builds up the string table of entries that require localization
 */
- (void)_scanSourceCodeFiles:(NSArray *)sourceCodePaths
{
    // create the aggregator
    DTLocalizableStringAggregator *localizableStringAggregator = 
                                    [[DTLocalizableStringAggregator alloc] init];
 
    // feed all source files to the aggregator
    for (NSString *oneFile in sourceCodePaths)
    {
        NSURL *URL = [NSURL fileURLWithPath:oneFile];
        [localizableStringAggregator beginProcessingFile:URL];
    }
 
    // wait for completion
    NSArray *stringTables = [localizableStringAggregator aggregatedStringTables];
 
    STAssertTrue([stringTables count]==1, @"Only one string table supported");
 
    _localizableTable = [stringTables lastObject];
}

When this method is done processing you have a string table in the _localizableTable instance variable.

Parse ALL THE STRINGS

For this stage we could do something fancy like dynamically generated the test methods. But that’s besides the point of this article.

We are going to specify the path to the German and English Localizable.strings files relative to the project folder. Then we are going to parse these files and check off the localization macros we collected earlier. If at the end of the string file parsing we have any macros left then we know that these are missing a translation.

- (void)_testMissingLocalizationsForStringsFileAtPath:(NSString *)path
{
    // copy the macro strings into a lookup dictionary for this test
    _currentTestedStringFileDictionary = [[NSMutableDictionary alloc] init];
 
    // transfer entries to dictionary
    for (DTLocalizableStringEntry *oneEntry in _localizableTable.entries)
    {
        NSCharacterSet *quoteSet = [NSCharacterSet characterSetWithCharactersInString:@"\""];
        NSString *key = [[oneEntry cleanedKey] stringByTrimmingCharactersInSet:quoteSet];
        NSString *value = [[oneEntry cleanedValue] stringByTrimmingCharactersInSet:quoteSet];
 
        [_currentTestedStringFileDictionary setObject:value forKey:key];
    }
 
    // create URL for Localizable.string to test
    NSURL *stringsFileURL = [NSURL fileURLWithPath:path];
 
    // parse the strings file, this emits a parser:foundKey:value: for each token.
    // These encountered tokens we remove from the dictionary
    DTLocalizableStringsParser *parser = [[DTLocalizableStringsParser alloc] initWithFileURL:stringsFileURL];
    parser.delegate = self;
 
    STAssertTrue([parser parse], @"Failed to parse: %@", parser.parseError);
 
    // any left over tokens are missing localizations
    [_currentTestedStringFileDictionary enumerateKeysAndObjectsUsingBlock:^(NSString *key, NSString *obj, BOOL *stop) {
        BOOL isNoPlaceholder = ([obj rangeOfString:@"_"].length==0);
        STAssertTrue(isNoPlaceholder, @"Key %@ is not localized and is probably a placeholder", key);
    }];
}
 
- (void)testGermanStrings
{
    NSString *stringsPath = [[self _projectPath] stringByAppendingPathComponent:
                             @"LocalizationDemo/de.lproj/Localizable.strings"];
 
    [self _testMissingLocalizationsForStringsFileAtPath:stringsPath];
}
 
- (void)testEnglishStrings
{
    NSString *stringsPath = [[self _projectPath] stringByAppendingPathComponent:
                             @"LocalizationDemo/en.lproj/Localizable.strings"];
 
    [self _testMissingLocalizationsForStringsFileAtPath:stringsPath];
}
 
#pragma mark - DTLocalizableStringsParserDelegate
 
- (void)parser:(DTLocalizableStringsParser *)parser foundKey:(NSString *)key value:(NSString *)value
{
    [_currentTestedStringFileDictionary removeObjectForKey:key];
}

The two actual unit test cases are testGermanStrings and testEnglishStrings. Both call the _testMissingLocalizationsForStringsFileAtPath: passing the path to their respective strings file.

Parsing of the strings file emits a delegate callback and each such found token we know as having a translation. In the end we have another level of filtering, in this example a token needs to contain the underscore character to be identified as a place holder. This still allows for lazy localizations where the English token key is the same as the English translation.

Et voila! Running the unit test gives us the result we are looking for:

Here we are basically done with the core functionality of our unit test. Since we are scanning the source code files whenever the unit test launches we don’t have to worry about adding some dependency.

Of course you may let your creativity run wild and implement things like automatic detection of which localizations are present. Let us hear your ideas and creative approaches in the comments.

Conclusion

Having the proposed Localization Completeness Unit Test run for every development build would probably be way too annoying. So you would probably have it only run before critical stages like a new release or ad hoc to create a list of items that your translators should look at.

One way or the other I think that it a worthy goal to always be 100% certain that you are not shipping incompletely localized apps.

My thanks go to my colleague Stefan Gugarel who implemented this very solution in one of our largest apps.

Categories: Recipes