[macrophyllum]

Hi,

I have a few application which need to save thousands of objects in arrays. What's more, is that each time the user uses the program they will likely add more objects to different arrays, making the data structure larger and larger and thus writing the file to disk will take longer and longer.

I have noticed that iCal save events, as soon as the user creates them, the to a .ics files, but you never notice it saving. Maybe they have another thread do the saving, or maybe there is a way to just re-write part of the file?

Any thoughts, help or tips are most welcome.

~Ian

[Arkham_c]

Originally posted by macrophyllum:
Hi,

I have a few application which need to save thousands of objects in arrays. What's more, is that each time the user uses the program they will likely add more objects to different arrays, making the data structure larger and larger and thus writing the file to disk will take longer and longer.

Sounds like you might want to consider a database.

[macrophyllum]

But how does iCal do it? They certainly don't use a database.

[techtrucker]

Perhaps using NSArchiver and auto-saving every time an array or dictionary is added?

[smeger]

You could do something like what (I assume) Entourage X does. Store both the entry and its array position in your flat file. When an entry is changed, if the new entry uses the same number of bytes as the old one, replace the existing record. If not, mark the old entry as invalid and append the new entry to the end of the flat file. When a new entry is inserted into the middle of the array, just append the entry to the end of your flat file (it contains the location, so no problem). When an entry is removed, mark it is invalid in your flat file.

Periodically, you'll need to compact the flat file to remove the invalid entries for things that were removed.

[macrophyllum]

I am not so good with terminology. What is a "flat file", do you mean the file that will be read and written to disk?

[smeger]

Originally posted by macrophyllum:
I am not so good with terminology. What is a "flat file", do you mean the file that will be read and written to disk?

Yeah, as opposed to a directory or something like a resource-fork file.