NSScanner is designed more along the lines of sscanf(); i.e. to be able to scan scalar integers/floats/etc from a string. In that sense it does a lot more than StringTokenizer, though it is a bit lower-level.
If you just want to tokenize based on a substring, then NSString has a method called -componentsSeparatedByString: that will return an NSArray of all the tokens between occurrences of the specified substring. [someString componentsSeparatedByString:@"\n"] will break a string up into lines, for example. A truly wonderful method. (There's a corresponding -componentsJoinedByString: method on NSArray to put things back together.) Once you have an NSArray, you can just iterate over that.
Now, this tokenizes based on substrings, not character sets like StringTokenizer does. If you want that, you can put the following category in your (Objective-C) project to add a couple of methods to NSString:
// NSString+MyAdditions.h
#import <Foundation/NSString.h>
@interface NSString (MyAdditions)
- (NSArray *)componentsSeparatedBySeriesOfCharactersFromSet

NSCharacterSet *)aSet;
- (NSArray *)wordArray;
@end
// NSString+MyAdditions.m
#import "NSString+MyAdditions.h"
#import <Foundation/NSScanner.h>
#import <Foundation/NSCharacterSet.h>
#import <Foundation/NSArray.h>
// (or just <Foundation/Foundation.h> and be done with it :-) )
@implementation NSString (MyAdditions)
- (NSArray *)componentsSeparatedBySeriesOfCharactersFromSet

NSCharacterSet *)aSet
{
NSScanner *scanner = [NSScanner scannerWithString:self];
NSMutableArray *stringArray = [NSMutableArray array];
NSString *betweenString;
[scanner setCharactersToBeSkipped:nil];
while (![scanner isAtEnd])
{
if ([scanner scanUpToCharactersFromSet:aSet intoString:&betweenString])
[stringArray addObject:betweenString];
else
[stringArray addObject:@""]; // can only happen first time
if ([scanner scanCharactersFromSet:aSet intoString:NULL])
{
if ([scanner isAtEnd])
[stringArray addObject:@""]; // can only happen last time
}
}
return stringArray;
}
/*"
* Returns an array of all the "words" in the receiver, a word being any
* run of non-whitespace characters.
"*/
- (NSArray *)wordArray
{
NSCharacterSet *spaceSet = [NSCharacterSet characterSetWithCharactersInString:@" \t\n\r\v\f"];
NSCharacterSet *nonspaceSet = [spaceSet invertedSet];
NSMutableArray *wordArray = [NSMutableArray array];
NSScanner *scanner = [NSScanner scannerWithString:self];
NSString *aWord;
[scanner setCharactersToBeSkipped:spaceSet];
while (![scanner isAtEnd])
{
if ([scanner scanCharactersFromSet:nonspaceSet intoString:&aWord])
[wordArray addObject:aWord];
}
return wordArray;
}
@end
Those two methods are now part of the NSString class, and can be called just like -componentsSeparatedByString: (be sure to #import your category header if you use them else the compiler will give you a warning).
A couple of notes -- if there are leading or trailing whitespace characters, -wordArray will ignore them while the other method will put an empty string at the beginning/end of the returned array to indicate there was a separator at the beginning/end (consistent with componentsSeparatedByString). If you want separate based on whitespace, I tend to prefer using [NSCharacterSet characterSetWithCharactersInString:@" \t\n\r\v\f"] instead of the provided [NSCharacterSet whitespaceAndNewlineCharacterSet], as the latter doesn't include carriage return (\r) characters.
I took the above methods out of some other code; I may have made some typos in the transition to this posting.
And feel free to rename componentsSeparatedBySeriesOfCharactersFromSet into something that's less of a mouthful :-)