[fortepianissimo]

One note in John Siracusa's excellent Tiger review (http://arstechnica.com/reviews/os/macosx-10.4.ars) caught my eye:

"Mail now saves all of its messages in individual files in order to allow Spotlight to index them more easily. Earlier versions of Mail sometimes (depending on the account time) stored multiple e-mail messages in a single file. I have to wonder if the Mail data folder will become a huge sea of tiny files after a few years of use...and if it does, is there any problem with that? Most file systems tend to get angry when a single directory has more than, say, 100,000 files in it."

http://arstechnica.com/reviews/os/macosx-10.4.ars/20

This indeed sounds like a potentially huge problem. From my understanding of Spotlight, as long as there's a meta data importer responsible for a certain type of file IO, the changes made to the data of an app should automatically show up in Spotlight searches - it doesn't need to be on a per-file basis. For example, if you change a certain part of a big mbox file, an importer should be able to update just the right part of Spotlight index, thus Mail should be able to still use a more efficient database format for storage? Why changing it into this file-based storage approach?

[TETENAL]

If you store 100.000 mails in one file Spotlight would still find the contents in each and every mail. But it would only know that the mail is in this large mailbox file. Not too helpful. If each mail is one file Spotlight finds the file and therefore each individual mail. It can show title and excerpt and it can directly open the particular mail.

I'm sure they tested whether it would be an issue for the filesystem.

[fortepianissimo]

Originally Posted by TETENAL

If you store 100.000 mails in one file Spotlight would still find the contents in each and every mail. But it would only know that the mail is in this large mailbox file. Not too helpful. If each mail is one file Spotlight finds the file and therefore each individual mail. It can show title and excerpt and it can directly open the particular mail.

So the "importer" can't do something smarter to tell Spotlight that a particular word comes from a particular email (instead of just pointing to a file)?

Or is it because there is no plugin architecture for Spotlight to present search results for non-file objects?

For example, with a database-based Mail.app, the importer can say word "Apple" comes from email 12345 (and possibly others). When a user searches for "Apple", Spotlight would know email 12345 matches.

Since email 12345 is an "email object" (not a file object), Spotlight then uses another plugin to present this result in its window. When a user double clicks on this object, the right email will show up in Mail.app.