Tokenize This!

I have two vendor IDs, one from my time as Individual and the newer one since I changed to a Company developer account. There might be other other reasons for having multiple vendor IDs, if you know of any, please let me know. I’m curious.

Now for this week’s update to AutoIngest for Mac I wanted to turn the plain text Vendor ID field into a token field like the Mac mail app has. Fortunately NSTokenField exists on Mac since 10.4 and today we shall explore its use.

Googling for a tutorial did not yield any immediate result, but I found a sample from Apple themselves which I am using for pointers.

First step is to go into the preferences controller’s XIB and replace the NSTextField with an NSTokenField.

Connect the token field’s delegate, also we add an outlet to our file’s owner so that we interact with it. I set the Layout to “Scrolls” and checked “Uses Single Line Mode” to have the token field a single line and scrolling horizontally if there are more than two tokens.

Even without any additional delegate methods implemented we already get some basic behavior. You can enter text and as soon as you type a comma, the text turns into a token. Now you know why they call it the “blue pill”.

The token field continuously check its contents and separates tokens by the set tokenizingCharacterSet. The best place for additional setup is the -awakeFromNib.

// any non-number is a seperator
NSMutableCharacterSet *characterSet = [NSMutableCharacterSet alphanumericCharacterSet];
[characterSet formUnionWithCharacterSet:[NSCharacterSet whitespaceAndNewlineCharacterSet]];
[characterSet formUnionWithCharacterSet:[NSCharacterSet punctuationCharacterSet]];
[characterSet removeCharactersInString:@"1234567890"];
 
self.vendorTokenField.tokenizingCharacterSet = characterSet;

NSTokenField has a class method defaultTokenizingCharacterSet, which you can overwrite in a subclass if you want to keep your token code in a reusable sub-class of NSTokenField.

There are several methods in the NSTokenFieldDelegate protocol which allows us to a few customizations.

Token Styles

You can think of a token field as an array where all elements are tokens. A token can either be a text string or the above mentioned “blue pill”. So if you want to have tokens inside some normal plain text, then the normal text would be represented by NSPlainTextTokenStyle whereas the actual tokens would be using the NSRoundedTokenStyle.

In its simplest form you would be using NSString instances for the individual tokens. To achieve the differentiation between plain text and rounded token styles you would employ a delegate method:

- (NSTokenStyle)tokenField:(NSTokenField *)tokenField styleForRepresentedObject:(id)representedObject
{
	// make valid vendor ids "blue pill", everything else plain text
	if ([representedObject isValidVendorIdentifier])
	{
		return NSRoundedTokenStyle;
	}
 
	return NSPlainTextTokenStyle;
}

The representedObject – as often used on AppKit – can be any object that should represent the tokens. The default implementation is to use the NSString values you enter, but you can easily provide your own backing objects.

Represented Objects

If you use represented objects instead of the default strings, then you have to implement several delegate methods because the token field needs to convert between the object and what it should write on the token and what the editing value should be. From the NSTokenField.h header:

// If you return nil or don't implement these delegate methods, we will assume
// editing string = display string = represented object
- (NSString *)tokenField:(NSTokenField *)tokenField displayStringForRepresentedObject:(id)representedObject;
- (NSString *)tokenField:(NSTokenField *)tokenField editingStringForRepresentedObject:(id)representedObject;
- (id)tokenField:(NSTokenField *)tokenField representedObjectForEditingString: (NSString *)editingString;

You provide backing object for a given editing string. Conversely if the user double-clicks on a token this turns into editable text. Finally the display string is the inscription on the blue pills.

For example if you have the token represent an email address, then the editing string could be “Oliver Drobnik <oliver@cocoanetics.com>” and the display string be just “Oliver Drobnik”. In that case you could have a token object class with a displayName and an email string.

Autocompletion

NSTokenField also supports showing suggestions to the user depending on some initially typed characters. Those are shown after the completionDelay which you can customize. In the case of e-mail addresses you could show the most likely completions entered so far and set the index of the most likely candidate. The defaultCompletionDelay – also a class method you can override via sub-class – is 0. When trying out different values I couldn’t notice any difference however.

Autocompletion values are provided with this delegate method.

- (NSArray *)tokenField:(NSTokenField *)tokenField completionsForSubstring:(NSString *)substring indexOfToken:(NSInteger)tokenIndex indexOfSelectedItem:(NSInteger *)selectedIndex
{
	*selectedIndex = 1;
	return @[@"8xxxxxxxx", @"81234567"];
}

If you also change the by-ref selectedIndex then the selection at this index will already be put into the text field. To avoid this you have to return -1. Then you only see the suggestions dangling below the field waiting for the user to pick one. Not setting the value is the same as returning 0.

I very briefly considered storing valid vendor IDs in the user defaults and then presenting these to the user for auto-completion, but decided against that.

Pasteboard

You might remember from the mail.app that you can drag around the e-mail tokens or even drag them into other apps. To support interaction with the Mac pasteboard you have to implement two methods.

The first method gets an array of represented token objects and lets you add some useful representation(s) to the dragging pasteboard. The objects you set there need to be able to implement the NSPasteboardWriting protocol. Alternatively you can create a property list representation for them  and set this instead of an array of objects.

- (BOOL)tokenField:(NSTokenField *)tokenField writeRepresentedObjects:(NSArray *)objects toPasteboard:(NSPasteboard *)pboard
{
	[pboard writeObjects:objects];
	return YES;
}

If you don’t implement this method then it still works, the default representation is to concatenate all tokens into a single comma-separated string. Since we are only using NSStrings as token objects we don’t have to do anything special since NSStrings know how to represent themselves on the pasteboard. This implementation causes the tokens to be staying individual values. If you drag&drop them for example into Notepad they come out as one per line.

Context Menu

Apple’s sample also shows how to add individual context menus to the tokens. This is achieved via the following two delegate methods.

// By default the tokens have no menu.
- (NSMenu *)tokenField:(NSTokenField *)tokenField menuForRepresentedObject:(id)representedObject;
- (BOOL)tokenField:(NSTokenField *)tokenField hasMenuForRepresentedObject:(id)representedObject;

This context menu is possible in addition to auto-completion and is available in the tokenized form. It’s basically a drop down menu with a down arrow to open it. The sample shows a nice trick how to force a token back into editing mode. This action is tied to an Edit… menu option.

- (IBAction)editCellAction:(id)sender
{
	NSText *fieldEditor = [self.tokensField currentEditor];
	NSRange textRange = [fieldEditor selectedRange];
 
	NSString *replacedString = [NSString stringWithString:menuToken.name];
	[fieldEditor replaceCharactersInRange:textRange withString:replacedString];
	[fieldEditor setSelectedRange:NSMakeRange(textRange.location, [replacedString length])];
}

Though I fail to see the practical use of this trick, since you can always go into edit mode by double-clicking on a blue pill token.

Validation

We already have a bit of validation going by only turning valid vendor IDs into pills. But there is yet another delegate method which would also allow us to completely reject invalid tokens.

The following only accepts valid tokens. This also removes invalid tokens if you enter a separator or tab out of the field.

- (NSArray *)tokenField:(NSTokenField *)tokenField shouldAddObjects:(NSArray *)tokens atIndex:(NSUInteger)index
{
	NSMutableArray *validTokens = [NSMutableArray array];
 
	for (NSString *oneToken in tokens)
	{
		if ([oneToken isValidVendorIdentifier])
		{
			[validTokens addObject:oneToken];
		}
	}
 
	return validTokens;
}

Depending on your use case you might not want to have invalid tokens simply disappear. If you want to keep them, but just have them be text, then you just forego implementing this delegate method.

A second method to only get validated values into the array of vendor IDs is to have an NSValueTransformer which gets the array of tokens, filters for valid and sorts uniquely.

@implementation ValidVendorValueTransformer
 
+ (Class)transformedValueClass
{
	return [NSArray class];
}
 
- (id)transformedValue:(id)value
{
	NSArray *array = nil;
 
	if ([value isKindOfClass:[NSArray class]])
	{
		array = value;
	}
	else if ([value isKindOfClass:[NSString class]])
	{
		array = @[value];
	}
 
	NSAssert(array, @"Invalid class for transformer");
 
	// make unique in set
	NSMutableSet *validTokens = [NSMutableSet set];
 
	for (NSString *oneToken in array)
	{
		if ([oneToken isValidVendorIdentifier])
		{
			[validTokens addObject:oneToken];
		}
	}
 
	NSSortDescriptor *sort = [NSSortDescriptor sortDescriptorWithKey:nil
                                  ascending:YES
                                  selector:@selector(compare:)];
	return [validTokens sortedArrayUsingDescriptors:@[sort]];
}
 
@end

This also has a provision for the vendor ID value previously being a string. With this it automatically migrates the value next time you enter the preferences. You install the value transformer in the data bindings inspector for the token field.

Also note the “Continuosly Updates Value” which I set so that whenever the contents changes the user default gets updated.

Conclusion

I wanted to achieve the effect of a valid vendor identifier automatically turning into a rounded token, but failed. I was able to see that the token field would turn valid IDs into my own VendorIdentifier represented objects, but it didn’t update the display. Only when I entered a separator character or tabbed out of the token field would I see the refresh occur.

There are no other visual customization options – that I could find – on NSTokenField so you have to be content with the “blue pill” style. But once you wrapped your head around the basic concepts and accepted the limitations of the API you can quickly use token fields to add them to your apps.

They work nicely intermixed with normal text or as “token only” fields which you can data-bind to an NSArray.

Categories: Recipes