[theolein]

Originally posted by JCS:
<STRONG>

Applications use XML-based .plist files, yes. (Although note that they continue to needlessly require ".app" extensions on the enclosing folder as well.) But there is no indication that XML will play any role in the future of file metadata in OS X. Instead, file name extensions are the recommended solution for the future (with type/creator supported to ease Mac OS 9 compatibility).

Does that sound like a strategy befitting "the world's most advanced operating system" to you?</STRONG>

I agree that it isn't the official path. It seemed to be a good idea that is all. The only problems with metadata on files, all metadata is that they add to system overhead, and Apple already had bad enough headaches with system overhead on the classic MacOS. However I don't really think there is any real way around this. I don't think Apple will budge from the Unix compatibilty of the underlying OS, although they might surprise us all in the end.

My suggestion is that Apple implement something like the old desktop database, but based on XML, where all the extra info you want can be stored on files and which will obviously be lost when moving files to other OS's like Windows etc. The standard Unix tools of course wouldn't support this but Apple already has ditto and CpMac for this, plus the finder of course.

A stop gap measure could also be for the finder to implement plists for files as well as applications, but you can imagine the problems involved here as well as they system overhead. I suppose if you has a sort of parse daemon running all the time you could get it a bit faster, but it would still be slow.

I don't know, I think a desktop database solution would be the fastest, but as I said I don't think it will happen. Apple has already had enough trouble getting people to port and write applications for MacOSX and if you look at all the time that has been spent on doing this and the trouble and bugs that have been involved with trying to merge the classic MacOS way of doing things and the NeXT way of doing things I seriously doubt there will be much chnage here in the near future. I think much in the same way that carbon seems to be becoming the default API on MacOSX, although it was intended as a temporary solution, I think the classic Type/Creator info will still be used by many people for a long long time to come.

[JCS]

The only problems with metadata on files, all metadata is that they add to system overhead

See my rant (previously linked in this thread) for my position on that.

And, honestly, the performance overhead is nearly meaningless. After all, if an 8MHz machine with 128*K* of RAM could somehow handle reading and writing a little bit of "extra" metadata to 400K floppy disks, call me crazy if I'm not shedding any tears for OS X on a G4 in this area

[Ulf Dahlen]

Originally posted by zpincus:
<STRONG>Thank you ulf for clarifying an important point!


I was wondering why so many people were soviciously against metadata.... Looking back through these posts, it does seem like it's the resource forks that they do dislike. (I do to, for that matter!)

Again, great point. We need more of those around here.</STRONG>

I just like to state that I also dislike multi-forked files and I think it's a good idea that the Mac is moving away from that.

I'm a Unix guy and really like the application-is-a-directory approach of Mac OS X. It is not perfect (e.g., you can have a file "xxx" and an app-dir "xxx.app" that looks the same in the Finder, which is confusing), but I think the reward is worth the price. You can easily traverse and edit the files (resources) in an application, both from the Finder and from a Unix shell.

However, for a community that does "chmod", "chown", "chgrp" on a daily basis to refuse to do "chtyp" and "chcre" seems rather silly. Type and creator are simply another piece of information, not unlike owner, group and permissions. It helps keep the filename clean (which is simply wonderful for the user) and it makes application binding easier.

What is missing is a good user interface for type and creator. I use "XRay" for Mac OS X, a very good utility where you can change every meta-data imaginable. And it's a contextual menu item (right-click on a file and select XRay). But this should be standard, with added features like multiple-item change.

For compaitibility, I think the binding process in Mac OS X is good. It first checks for a special, direct binding, then creator code, then file extension and lastly file type. Keep this.

What I strongly object to is Apple releasing software that doesn't set the type and creator code and urging others to do the same. Why? The Mac can handle "foreign" files, and, since there are no resource forks, foreign systems can handle mac files (just add the proper extension). So, interoperaibility is no problem. But why remove features from the Mac? It makes no sense at all.

I'm all for making the Mac work in a network with other systems, such as Windows and Unix, but there is no reason to remove features from the Mac when it clearly isn't necessary. And file extensions are such an ugly and awkard concept that even Microsoft have gone to great lengths to make them invisible to the user.

[cpt kangarooski]

I disagree. I prefer multiforked or multistreamed files because they seem to be more robust (i.e. less prone to turning into folders) and can be easily converted into truly seperate files if moved to filesystems that can't support them. Given that NTFS natively supports streams, and support is reportedly coming in Linux, and Apple has had portability mechanisms like PCExchange for around a decade, compatability arguments are pretty worn out.

Of course, what's missing from either implementation is a good UI for working with them. Apple really has to wake up and realize that a single standard folder UI is no longer a viable solution. XP is doing far better in this regard, although it's not all that customizable, which is bad. It would be nice to give Apps and multiforked Files behaviors like the MacOS System Folder, where you can drag on things like plug ins and have them properly install themselves. And add a far better UI, perhaps distantly related to Extensions Manager, for intermediate users to mess with the contents. (a more thorough folder like UI might be available, but it's so lame that I can't imagine it being used much)

The problem with forked files is NOTHING inherent to them, it's merely that Apple has basically not improved MacOS significantly since System 7.1. Don't blame the technology when it's not its fault!

[Mr Scruff]

Originally posted by Ulf Dahlen:
<STRONG>What I strongly object to is Apple releasing software that doesn't set the type and creator code and urging others to do the same. Why? The Mac can handle "foreign" files, and, since there are no resource forks, foreign systems can handle mac files (just add the proper extension). So, interoperaibility is no problem. But why remove features from the Mac? It makes no sense at all.

I'm all for making the Mac work in a network with other systems, such as Windows and Unix, but there is no reason to remove features from the Mac when it clearly isn't necessary. And file extensions are such an ugly and awkard concept that even Microsoft have gone to great lengths to make them invisible to the user.</STRONG>

I agree 100%. There simply is no need to remove creator/types from the Mac OS. It won't enhance compatibility with Windows one bit. It simply removes existing functionality.

[Agent69]

Although I personally dislike Metadata, I would be happy if Apple would just go one way or the other on this. It's stupid to have some apps making use of it and others not. Even though I don't care for it, at least the older Mac OSes were consistant.


Agent69

[Developer]

Agent69, would you mind to explain? How can one dislike meta data?

Creation date, modification date, privileges, labels etc. pp. You dislike those all?

[Agent69]

I don't think the Mac OS metadata is worth the effort put into it. A filename extentsion is enough and is more in line with what other OSes use. I don't see the point in lables, comments, resource/data forks, or any of the other stuff the classic MacOS uses.

Still, that's my opinion and I can live with using the Mac metadata if it is used in a consistant manner, and it isn't right now. Some apps use it, some don't, and I think that it is making things hard on some users.

Agent69

[zpincus]

Originally posted by Agent69:
<STRONG>I don't think the Mac OS metadata is worth the effort put into it. A filename extentsion is enough and is more in line with what other OSes use. I don't see the point in lables, comments, resource/data forks, or any of the other stuff the classic MacOS uses.

Still, that's my opinion and I can live with using the Mac metadata if it is used in a consistant manner, and it isn't right now. Some apps use it, some don't, and I think that it is making things hard on some users.

Agent69 </STRONG>

I'm glad to see a reasonable approach to the issue...

However, I believe that being "more in line with what other OSs use" is pretty inimical to the reason I'm using a mac in the first place. I use a mac because it offers a richer experience than other platforms. When the choice is dumbing down my Mac experience versus simply putting in the work to ensure that every file that leaves my computer gets tagged with the right file extension, I choose the latter. (Both approaches make the mac compatible with other platforme, file-wise, abd both require tha Apple make developers follow certain guidelines about file names.)

Also, I really do suggest that you try deleting the ".rtfd" from a RTFD file and ask yourself if file extensions are really enough... (hint: files should not loose their associations, or worse, turn into folders, just because their name gets messed up [which happens with high frequency...])

You really should give John Sircusa's article a read -- its fascinating, and it clued me into a lot of things. (Free preview: resource forks are not metadata, but are in fact one of the reasons that "metadata" gets a bad name.)

[Brass]

Originally posted by Agent69:
<STRONG>I don't think the Mac OS metadata is worth the effort put into it. A filename extentsion is enough and is more in line with what other OSes use. I don't see the point in lables, comments, resource/data forks, or any of the other stuff the classic MacOS uses.

Still, that's my opinion and I can live with using the Mac metadata if it is used in a consistant manner, and it isn't right now. Some apps use it, some don't, and I think that it is making things hard on some users.

Agent69 </STRONG>

NB: RESOURCE FORKS ARE NOT METADATA!!!

excuse me for shouting. It just bothers me that most of the "anti-metadata" posts here (and some of the pro-metadata posts) don't seem to understand this.

NB: TYPE&gt;CREATOR CODES ARE NOT STORED IN THE RESOURCE FORK!!!

excuse me again. Similar point but again, most people seem to get confused between metadata and resource forks. They are completely different.


I'm very glad to see resource forks go. This way I can upload/download files across networks between different platforms and not lose anything from the file at all.

However, I don't want to lose the filetype metadata! I can see that people want to use file extensions for cross-platform compatibility, that makes sense. But why force me to use it on all files all the time? Why not just have a standard by which the OS recognises both type/creator codes AND file extensions (putting the higher priority on type/creator codes - or perhaps user configurable?). Along with a policy of having all network transfer applications (ie, email, web browser, etc) convert between type/creator and file extension every time a file is passed to/from the network.

This would be so simple to implement, and would be so functional!

[Agent69]

I stand corrected then. I was my understanding that the resource fork contained (among other things) the creator/type information, and that it was considered to be metadata.


Agent69

[Agent69]

Originally posted by Brass:
<STRONG>

Why not just have a standard by which the OS recognises both type/creator codes AND file extensions (putting the higher priority on type/creator codes - or perhaps user configurable?). Along with a policy of having all network transfer applications (ie, email, web browser, etc) convert between type/creator and file extension every time a file is passed to/from the network.

This would be so simple to implement, and would be so functional!</STRONG>

This sounds like something that BeOS does. BeOS stores file types (based on the MIME standard) as an attribute of a file. Even if you delete the extension of a file, BeOS can open it if the attribute has been set.


Agent69

[Millennium]

Originally posted by moreno:
<STRONG>siracusa if you love metadata, don't switch to mac os X...
we need evolution, and metadata only makes the things difficult...</STRONG>

Evolution by stagnation? Um, load of crap, much?

Evolution means change, difference. It is by using metadata that we evolve, not by going back to an inferior standard which was basically an uggly, ugly hack that has no place in a modern GUI.

OS9's filetype implementation had its flaws, but it got one thing right: extensions were the last resort, not the default, and this fact was made clear to programmers. OSX should build off of this base, not ditch it. Interoperabilitty is a part of usability, but just because we support a standard doesn't mean we have to rely on it ourselves. OS9 did this; there were problems, but it was better than what we have now. Even worse, adding the kinds of things needed (proper metadata in Cocoa, calls to associate an extension with a T&C pair, etc.) shouldn't even be all that difficult; all that is needed in most cases are wrappers around existing Carbon code or simple calls to the type database.

This is not a simple interface issue like labels or WindowShade or the like. In those cases, Apple has replaced the old functionality with new alternatives, or is working on re-creating the old (with evidence to support that claim firmly in place). But they didn't do that with this. There is no suitable replacement for T&C at hand, and while I don't doubt that Apple is probably working on a suitable replacement, they should at least support the old until the new is ready.

[Agent69]

Originally posted by Millennium:
<STRONG>
Evolution by stagnation? Um, load of crap, much?
least support the old until the new is ready.
</STRONG>

True, but change for change's sake isn't always good either.


Agent69

[Mediaman_12]

I also read along whith this 'keep T&C Metadata petition' there in now an 'anti T&C Metadata petition'.
&lt;sarcasam&gt;
Well done to the people who started that one.
&lt;/sarcasam&gt;
What we have now is the likley hood that each will cancel each other out.
All people wanted was a choice. If you wanted to use the .3 letter suffexes you could, but if you didn't you wouldn't end up in trouble.
IMHO 3 letter sufexes suck anway seeThis site for a small selection of the reasons why.
I thought Unix could have more than 3 letters for the filetypes anyway?

[Agent69]

They can.


Agent69

[nickm]

I thought Unix could have more than 3 letters for the filetypes anyway?

Vanilla Unix doesn't support file type metadata in any meaningful way. The main interface for working with files, the command-line tools ls, mv, and cp, are agnostic about the "type" of a file. There are no standard command-line programs or OS system calls for setting or getting the type of a file. Contrast this to other metadata that UNIX does care about, such as file permissions, owners, and dates. There are command line tools (chmod, chown, touch) and OS system calls for dealing with this type of metadata.

This isn't to say that some UNIX-based applications don't use file names extensions to determine the type of the file, but this is different than support at the level of the operating system. What this petition is asking for is adding this functionality at the level of the OS, so that all applications can share the same conventions for accessing file types, ensuring that different applications can deal intelligently with the same files.

[dont.wanna.tell]

Salve!

-- First: Sorry, this is kinda long. If you dont want to read it just scroll down to the bottom of it. The last part is the most important. --

I dont know how you are using your computer, but I do it in a way that I just want to like it. (at least most of the time)

So I dont like to see the little spots at the end of every file. BUT I also dont ever want to get caught in this (at least in the last time within os X) quite common situation that the Computer simply does something different whith the files than I wanted it to do.

So what to do? The other halv of myself is technical quit firm and nows whats happening in there and can go down to the little bits to fix everything myself. But I dont want to.

I do think that that is the perfect thing for a computer to do. This has not much to do with type and creator codes, it has not much to do with appending file types to the end of files, it has simply something to do with wanting to work with my computer without having to deal with stuff like that when I dont like to do it.

So I dont really care how the solution to this problem will be, for me either Type and Creator Codes as file information in the file name are still verry clumsy "tries" to tackle the problem.

The only problem in this is that Apple seems just so stuffed up about this and it simply seems that they want us to stick with such a badly done implementation as the only way of doing it.

(Of course sometimes my other part is in command and I really do want to have file extensions, but then, nobody hinders me to get them.)

I dont know if this has ever been done in the past. I *think* that Be was pretty close to achieve this sort of thing. But then, thats history.

What I would think of as an end all to this discussion would be some sort of statement from Apple that they are actively developping a solution. For the start an enhancemend (look at Johns article for an excelent idea how this could be done), and are also getting into the problem that we do need a even more modern File system than hfs+ that also does modern things like journaling (then I'le never have to wory about a crash again! yikes), automatic optimization (no Norton anymore) and is also fast. And please dont tell me that this aint possible, just look out how many file systems have been developped, just buy one and make it an option on install time. Thats enought to please users like me. ))

cu Martin

[Brass]

Just to supplement my earlier post where I stated that file type/creator metadata was not stored in the resource fork...

The file type/creator metadata is actually stored as part of the filesystem itself, the same as timestamp metadata. There are two reasons that Apple appears to be dropping type/creator metadata. One of course is that no other OS understands it (I think this is a BAD reason - heading towards the mediocrity of other systems).

The other reason is that no other FILE SYSTEM stores this exact type of metadata (only HFS and HFS plus), and now Apple want to support other FS types, especially UFS (the unix standard file system). However, that does not mean that type/creator metadata cannot be stored on other filesystems. It just cannot be stored in the same way on other filesystem. Apple are already using the (annoying!) ".DS_Store" files for all sorts of metadata (including position of icons in windows). Why not continue this concept?

I think the ".DS_Store" concept is slightly flawed. Mainly because I don't want these metadata files getting packaged up and sent off with the directory when I zip/stuff a directory to send to a non-mac user. However, there's no reason why a similar scheme could not be implemented. Eg, have all the .DS_Store files in another directory, such as /DS_Store. So the .DS_Store file for the directory /Users/brass/documents could actually be stored as the file /DS_Store/Users/brass/documents.

Of course this scheme has it's own flaws (deleting directories?), but they could be worked out, or perhaps there is a better solution again.

[sadie]

Originally posted by theolein:
<STRONG>Apple is moving towards an XML based solution of storing Meta Data on files and applications. What this means, as far as I can tell, is that Apple is bothmoving away from type/creator and file extensions.

...I think we are maybe worrying a bit to much about this as it seems that there is a good Meta data solution coming in any case.</STRONG>

Uhh... I hate to point this out to you, but XML is (quite deliberately) such an extensible format that "an XML based solution" does absolutely nothing to narrow down what that solution actually it. Type/creator codes could be an XML-based solution, as could file names and all sorts of other things.

Do you know what it really is? Or do you have blind faith in an XML-based solution?

The danger is that the solution will be so intricately linked into the innars of .plists that we can't easily work out what's going on. The vital benefit of both type/creators and file extensions is that human beings can easily understand them. If the metadata gets too bogged down into the intricate XML formats scattered around the system, we'll lose that.

[cpt kangarooski]

brass--
NTFS understands it, or at least preserves it. At my previous job we had a Win2K fileserver that could store multiforked Mac files all day, could be mounted as under Apple File Sharing, and behaved perfectly.

Given that it's the official fs of the entire NT line, AND of XP, claims that Mac metadata and forks aren't easily portable are on shaky ground. More so, when you consider older but still viable additional methods such as splitting files as under PCExchange, or AppleDouble.

[hELLO wORLD]

Apple are already using the (annoying!) ".DS_Store" files for all sorts of metadata (including position of icons in windows). Why not continue this concept?

I think the ".DS_Store" concept is slightly flawed. Mainly because I don't want these metadata files getting packaged up and sent off with the directory when I zip/stuff a directory to send to a non-mac user. However, there's no reason why a similar scheme could not be implemented. Eg, have all the .DS_Store files in another directory, such as /DS_Store. So the .DS_Store file for the directory /Users/brass/documents could actually be stored as the file /DS_Store/Users/brass/documents.

Of course this scheme has it's own flaws (deleting directories?), but they could be worked out, or perhaps there is a better solution again.[/QB]

I can see something very important with this, and with metadata in general.
There should be file-metadata and user-metadata.
I will try to be more explicit:
Privileges are unique to the file, but some attributes should be differentiable by users.
For exemple, one file have one type and one creator, and this is unique, but a user should be able to setup its own creator, without altering the "master creator". Each user should be able to put a file visible or invisible, without changing other users parameters on this file.
User A can have the file visible while user B will have this file invisible.
The default parameters should be based on the root ones, for exemple.

And about that DS_Store, one user would have to have big icons in the "Computer" folder, while an other user would use list view with date ordering. Both should be able to have their own parameters without disturb each other. It is not the case now.

Metadata is a very important thing, and in the unix world, we have to go one step further, and have user-based metadatas !

This would be so amazing and powerfull !!!

[Eug]

I think the ".DS_Store" concept is slightly flawed. Mainly because I don't want these metadata files getting packaged up and sent off with the directory when I zip/stuff a directory to send to a non-mac user. However, there's no reason why a similar scheme could not be implemented. Eg, have all the .DS_Store files in another directory, such as /DS_Store. So the .DS_Store file for the directory /Users/brass/documents could actually be stored as the file /DS_Store/Users/brass/documents.

Am I correct to assume these are the ._xxxxxx.xxx files that are being stored by OS X on my Windows box over the network? Fortunately, my MP3 receiver (which streams MP3z off my Windoze box) seems to ignore the ._xxxxx.mp3 filez, but other appz will allow to me to attempt to open them and of course will just get confused.

[absmiths]

Oh please! No XML! XML is such an overused data format. What is it? Merely a heirarchical data structure stored in the worst possible format (usually Unicode text) requiring the slowest possible interpretation (parsing). Imagine a basic metadata for a simple file:

Immutable bit set
Creator 'AdOb'
Type 'File'

Binary [9 bytes, not 5, Apple uses 32 bits for type/creator, right?]:
FF AdOb File

XML [74 bytes, 148 if unicode!]:
&lt;Immutable&gt;yes&lt;/Immutable&gt;
&lt;Creator&gt;AdOb&lt;/Creator&gt;
&lt;Type&gt;File&lt;/Type&gt;

I for one don't care to increase the storage of metadata by 7-15 times just to make it human readable!

I work for a company that has done just that - bought into the XML craze and implemented an XML middleware component which does XML messaging. It sucks, is slow, and makes no more sense to the receiving end than a published binary structure.

Besides, human readability is not a valid requirement for this type of system since it is only useful during debugging/trouble shooting - and a tool to read the data for the user would be written in any case.

[ 12-11-2001: Message edited by: absmiths ]

[absmiths]

Originally posted by Eug:
<STRONG>Am I correct to assume these are the ._xxxxxx.xxx files that are being stored by OS X on my Windows box over the network? Fortunately, my MP3 receiver (which streams MP3z off my Windoze box) seems to ignore the ._xxxxx.mp3 filez, but other appz will allow to me to attempt to open them and of course will just get confused.</STRONG>

I haven't ever seen that - does any specific app create temp files like that?

[cpt kangarooski]

Heh. I've been saying the same thing for a while. But take it even further: the PATH of a file, or even existance on a per-user basis of a file, could be unique per user. Thus I could keep files on the desktop, and you could delete some of them, and keep the rest in a folder deep on the hd, and we'd both get along fine! (changes made to the files would either, depending on whether they were set to be shared or not, result in a change to the underlying file, or a fork, giving each user a totally seperate file rather than merely different metadata on the same file)

[theolein]

Originally posted by absmiths:
<STRONG>Oh please! No XML! XML is such an overused data format. What is it? Merely a heirarchical data structure stored in the worst possible format (usually Unicode text) requiring the slowest possible interpretation (parsing). Imagine a basic metadata for a simple file:

Immutable bit set
Creator 'AdOb'
Type 'File'

Binary [9 bytes, not 5, Apple uses 32 bits for type/creator, right?]:
FF AdOb File

XML [74 bytes, 148 if unicode!]:
&lt;Immutable&gt;yes&lt;/Immutable&gt;
&lt;Creator&gt;AdOb&lt;/Creator&gt;
&lt;Type&gt;File&lt;/Type&gt;

I for one don't care to increase the storage of metadata by 7-15 times just to make it human readable!

I work for a company that has done just that - bought into the XML craze and implemented an XML middleware component which does XML messaging. It sucks, is slow, and makes no more sense to the receiving end than a published binary structure.

Besides, human readability is not a valid requirement for this type of system since it is only useful during debugging/trouble shooting - and a tool to read the data for the user would be written in any case.

[ 12-11-2001: Message edited by: absmiths ]</STRONG>

You're absolutely right on this one. I thought about it myself as i was posting. The XML stuff I got out of Apple's own Dev Docs. XML would be ideal in that it is extensible, and certain parts of the file could be obligatory and Apple could leave the rest for developers to add funcionality to the Metadata. But as for parsing, well, it's slow and large.

But...

There is nothing stopping Apple from storing binary data in a hierarchial form that can be read by some precompiled XMLBinary parser or some such shlt. You know, just giving developers the instructions that the first section of bytes is for Apples use and cannot be changed, and the rest will be ignored by the FS unless there is some FS extension/Application that specifically uses the extra bits. There could be a control byte after the required bit to say that a certain application or extension has added data here and saves the extra data from being overwritten by yet another extension.

These applications could register themselves in some central registary (netinfo?) and each application could then use this transparently and not have to know that it was there unless it needed it. This would provide backward compatability and abve all would not interfer with the Unix underpinnings of OSX.

The key would be extensability. *If* the Finder for instance allowed extensions, an extension could show different versions of a file in a CVS like system or even something like a different icon for the same file for different users or differing user added notes. If a programm like project builder added an extension it would be the only one to use it and it's extensions would not be noticed say by bbedit or photoshop.

The problem of where to store all this has already been decided by Apple: .DS_store. If you su to root and do a more on this file you'll see that it is in fact a kind of plist with loads of binary data in it.(Look in a directory where there are lots of files not just other directories) So maybe all this has in fact been thought out by apple already and they're thinking "Why can't these people just be more patient?"

This is all stuff that I sucked up out of my imagination and by no means covers every angle or has any chance of being implemented but it would be nice if Apple did more thought on this.

[Eug]

Originally posted by absmiths:
<STRONG>

I haven't ever seen that - does any specific app create temp files like that?</STRONG>

Temp files???

Any file that gets moved to my Windows desktop over the network via SMB through the Finder gets saved with this type of extra file. Every single time.

[Developer]

Originally posted by Eug:

Any file that gets moved to my Windows desktop over the network via SMB through the Finder gets saved with this type of extra file. Every single time.

Is that on NTFS? I thought it supports multiple (i. e. in the case of Mac files the resource) forks. If so it would be kind of silly to split it to a seperate file (and pretty annoying too, since Windows doesn't keep them synchronized, so this would leave a lot of dead resource fork files around).

[ 12-12-2001: Message edited by: Developer ]

[Eug]

Originally posted by Developer:
<STRONG>

Is that on NTFS? I thought it supports multiple (i. e. in the case of Mac files the resource) forks. If so it would be kind of silly to split it to a seperate file (and pretty annoying too, since Windows doesn't keep them synchronized, so this would leave a lot of dead resource fork files around).

[ 12-12-2001: Message edited by: Developer ]</STRONG>

Not NTFS. This is Windows 2000, but I'm running FAT32. (At the time I initialized the drive, I was dual booting with Win 98 and wanted to be able to see the partition under Win 98.)

Would there be any point in reformatting? All I have on there at the moment are MP3 files and games, and a some backup files (45 GB dedicated drive). The only problem would be backing the thing up since I have probably about 20 GB of stuff on there, mostly MP3.

[Developer]

Originally posted by Eug:

Not NTFS. This is Windows 2000, but I'm running FAT32. (At the time I initialized the drive, I was dual booting with Win 98 and wanted to be able to see the partition under Win 98.)

Would there be any point in reformatting? All I have on there at the moment are MP3 files and games, and a some backup files (45 GB dedicated drive). The only problem would be backing the thing up since I have probably about 20 GB of stuff on there, mostly MP3.

Well, the point would be that we could see if it works. I have read that Windows can handle the Mac's resource fork just fine on NTFS. It just stores it - tadaa - in a separate fork.

If so (and maybe somewone with more knowledge than I have can shed some light on this) we can practically end this discussion now. NTFS is the default file system for Windows XP, so in 2 or 3 years it will be the default on most computers, and Mac OS X is build for the future, right?

So the only thing that remains would be compatibility to UFS. But I see no point in supporting a 30 years old single forked file system. As John Siracusa said in his proposal, the way Apple should go for compatibility shouldn't be the one of the lowest common denominator, but a file system that is a superset of existing file systems.

[cpt kangarooski]

It works. We had a Win2K machine set up at my old work that was mounted on the Macs like a normal server. Damn useful, as well. NTFS is a better fs than FAT32 anyway, it supports larger file sizes, its journaling, etc. I'd definately switch over unless compatability with older OSes is essential to you. (Humorously, Apple is still ahead of MS with backwards compatability with FAT32 -- There's no Windows equivalent to PC Exchange that splits multistreamed files into multiple flat files to preserve them)

As for Unix, as I've pointed out before, Linus has been vocal about wanting support for such features introduced by someone (not his job, but he's got a lot of cachet) if for no other reason than neighborliness. Yep, the future's sure looking like flat files, yessir.

[dont.wanna.tell]

So the bottom point is, that we all just want apple to clarify their points, why they did what they did, and where they want to go.

And this is exactly the reason of the poll. (: As stated in Johns article

So, head over and sign it. )

Martin

[Agent69]

NTFS does indeed support data forking but I think that it depends on how you create and access the file.

For example, when I used NT4 Server's Services for Macintosh (SAM; MS's Appletalk implementation) on my server and accessed that server via Mac OS X's Appletalk client, I didn't have all those extra resource files. But when I accessed the server via the SMB client, I did get them.

[NOTE: You cannot create files longer then 31 characters on a NT4 Server using Services for Macintosh. You will get an alert if you try to do so. Newer versions of Windows 2000 Server may support this.]

For those who wish to access a CIFS (SMB) network and not have a bunch of extra files, The Dave SMB client is probably your best bet, provided the server is running NTFS. From the FAQ:

&lt;snip&gt;
Q. Do I still need DAVE now that Apple has SMB built into Mac OS X version 10.1?

A. DAVE has some key features and functions that aren't available with Apple's offering. While Apple's SMB works only with a command line interface, DAVE uses a full browser to navigate the network.

Another very important difference is how each product stores their files on the remote system. On Microsoft systems supporting NTFS volumes, DAVE makes full use of this by storing both forks of the file within the single file. This is similar to, and fully compatible with, the methods used by Microsoft's Services for Macintosh. Apple's SMB stores the data in a manner that is incompatible with both DAVE and Microsoft's methods.
&lt;snip/&gt;


Agent69

[ 12-13-2001: Message edited by: Agent69 ]

[Brass]

Originally posted by cpt kangarooski:
<STRONG>brass--
NTFS understands it, or at least preserves it. At my previous job we had a Win2K fileserver that could store multiforked Mac files all day, could be mounted as under Apple File Sharing, and behaved perfectly.

Given that it's the official fs of the entire NT line, AND of XP, claims that Mac metadata and forks aren't easily portable are on shaky ground. More so, when you consider older but still viable additional methods such as splitting files as under PCExchange, or AppleDouble.</STRONG>

I disagree. Although you can copy Mac files to a Windows partition and preserve the file/creator type (ONLY if using some particular file transfer protocols, like AppleShare), this does not make it portable for two reasons: Windows ignores it anyway, even if it is there! The data is still lost if you use one of several standard file transfer mechanisms (eg, FTP, HTTP).

Of course the best solution to the metadata problem would be if Windows (the dominant player) started using Mac-style type/creator codes, over and above filename extensions. If that happened, then everyone else would eventually follow (Apple would then follow MS in their following Apple?). But if we think that is ever going to happen, then we're dreaming.

[Brass]

Originally posted by cpt kangarooski:
<STRONG>brass--
NTFS understands it, or at least preserves it. At my previous job we had a Win2K fileserver that could store multiforked Mac files all day, could be mounted as under Apple File Sharing, and behaved perfectly.</STRONG>

OOPS I misread your post at first!

Supporting Multi-Forked Mac files is not the issue here, because as I shouted in my earlier post, FILE AND CREATOR TYPES ARE NOT STORED IN THE RESOURCE FORK!!! They are stored by the filesystem, NOT in any part of the file itself, NOT in the data fork, NOT in the resource fork.

People keep confusing this issue.

That's not to say the NTFS doesn't handle TYPE/CREATOR codes, maybe it does, but that's nothing to do with handling resource forks.

[Agent69]

Although this is off topic, I would like to note that NT/2000/XP's support of data forks are a feature of NTFS and is called "Alternate Data Streams". I good explaination of this, for those interested, is at:
http://www.diamondcs.com.au/streams/streams.htm

Agent69

[hELLO wORLD]

More on my User-based metadata. :

Hi,
I will spend some time to be more explicit on my user-based meta-data idea.

I think the file itself should be stored somewhere on the harddisk, with minimal and important meta-data (permissions, type and maybe creator).
For each visible instance of that file (we mean here one instance per user), we will have user based metadata, like creator, visible, invisible, locked, etc... In fact, that instances would be like aliases with their own parameters (meta-data).
Of course, a power-user will be able to force some characteristics, like locked or permissions, and if the root decide to lock a file, avery user will have the file locked without the ability to unlock it (without authenticating as a administrator).
If a file is copied, it is the file itself that will be copied with original "master" characteristics, but when a user copy a file, the copied file will see this user meta-data copied too...

Here is an exemple :
There is a file called "FILE" with "master" meta-data with type : "TYPE", creator : "JOHN" and visible.
We have 3 users in this exemple : A, B and C.
The user A decide to set the creator of this file to "ERIK" because he prefers to link this file to the application "ERIK". This will not affect User B nor C : for them, creator will still be "JOHN"
The User B decide to put the file invisible, but it will still be visible for A and C.

User C puts creator to "KATE"
If the user C copies the file (copy filename is "COPY"), the master file will be duplicated on the disk, and the User-C meta-data for "FILE" will be appliqued to the duplicata file.
User A will have file "COPY" with default meta-data, and user B too.

The result is :
User A :
"FILE" : type "TYPE", creator "ERIK", visible
"COPY" : type "TYPE", creator "JOHN", visible
User B :
"FILE" : type "TYPE", creator "ERIK", invisible
"COPY" : type "TYPE", creator "JOHN", visible
User C :
"FILE" : type "TYPE", creator "KATE", visible
"COPY" : type "TYPE", creator "KATE", visible

Of course, users should be able to set preferences for one file, or all files of a specified type.

Mac OS X have some capabilities for that with DS_store files, and setup prefered app for a specific extension or a simple file, however, I am not sure this works for Creator.

I know this is not a finished projet, and there is maybe errors or things impossible to implement, but it is an idea, to start something that appears important in this modern system : the consideration of user choices, control and customization on files meta-data.

My 2 cents.

[cpt kangarooski]

Oh sure. Whether or not Windows _uses_ it is quite a different matter, from whether or not it preserves it. And for the record, it does preserve all of the metadata, as well as the dual forks -- NTFS has some slight advantages over HFS+'s spec, and some even more significant ones over HFS+ as it is actually implemented. I know the difference between the forks and m.d. and such.

However, with WinXP recognizing and USING ID3 metadata straight out of mp3 files, in the file manager, a feat which Apple has never accomplished, I strongly suspect that only a comparatively small amount of work would be needed for Windows to not only recognize type and creator codes, but to actually respect BOTH, and open identical files of the same type with different creators in appropriate, different apps. I bet a third party that was fairly conversant with the OS could do it. I'd lay money on it. It wouldn't be pretty, and might even have to be a total replacement of the file manager, and add some bloat to the registry, but I bet it's possible.

As for transfer mechanisms... who's fault is that? It's not as though _new_ versions of the various protocols don't crop up from time to time. Apple is equally at fault as anyone else there.

[Sjakelien]

Originally posted by JCS:
<STRONG>

But there is no indication that XML will play any role in the future of file metadata in OS X. Instead, file name extensions are the recommended solution for the future (with type/creator supported to ease Mac OS 9 compatibility). </STRONG>

I would like to point out that Adobe has developed an xml-based metadata protocol called XMP (take some time to read
this). I think it's the solution that Apple should embrace.
Basically it solves two problems:
1) The metatdata is stored in the data fork, so the information doesn't get lost during internet transfers
2) It's xml, so you can put whatever metadata in there, that's necessary for your needs. Not only Type/Creator stuff, but also copyright information, workflow status, you name it.
Adobe is incorporating this XMP information anyway in all documents created by the new versions of all their programs, so Apple might as well comply to this open approach.
File extensions are silly and old fashioned, even Microsoft nows that: I understood that metadata is going to big in their next (database-based) OS.
In short: sign the petition now!

[Mediaman_12]

This is the reason relying on filename extensions on there own is a mistake.

One of these files is an Audio file (an aiff from Spark me) the other is an Illustrator file. Both have .ai extensions . I have not altered the file names in any way. They are both still recognized by the different apps because they both are still using metadata.

[tazmandevil]

Listen! I tell you what! ... Youre all on the totally wrong way!.... (i don't believe it).... is Apple really on the one or the other side?... Why are you/they hitting each others head... to have the better side? Look: Both systems:
� Suffixes
� extra Metatags
are absolutely "THE WRONG WAY!!!".....
I tell you what!
Remember the "commodore Amiga Operating System".... there was a lot good ideas (but bad management).... often overseen... they had a "VISON".... similar to some "Steven Jobs" Visions.... (but one of them, was absolutely progressiv).... here it is:

� "Operating/Filesystems should not handle the FILETYPES anymore.... because FILETYPEs "ARE".... still available!!!!! Where?... YES, very simple:
"IN THE FILES ITSELFS".....!!!!!
-&gt; You just have to concentrate to structured DATAFORMATS, which idendifies them by herself!!!!! (like most do it today!)... look the first line, of every file!!!!! What can you read there?... Nothing as crypted Hexadecimal Sings?..... Wrong!!!!!!! (if so, its a idiotical format, but they are not often still used today).....
There you can read: "% PS2.0".... �#GIF"... and so far.... and often a lot more of Information, that some of you may call "META DATA".... this is still the PRESENT!!!!! -&gt; Structured Dataformats!.... (remember to the Amigas open IFF-Definitions.....

As a result of a disciplinating and concentrated Eye on Define Documents, Files and Archives....... it will be possilbe to "RECOCNICE and IDENTIFY" every file,
WHITOUT the two lacks: * SUFFIX or * METADATE(resources or something)....

The one IMPORTANT Point, of yours Arguments, to appoint APPLE to "do" something is true!!!:

To give the "USER" the full control about Filenames!!!!! (if they don't: soon we will have another kind of "Trillenium-Bug" in the near future!!!!!)....(think to the much linked Datas in the Press/Publishing Business... Think about, what happens, if Newspaper-Document "Headline" don't find it's own Headline Picture, because its Suffix is hiding or its Metadata is converted into something else *wu�hhh*))...(horror vision)!!

No, its really easy!

Programm the Filesystem this kind, that it no longer has to change anything on each file, but that it will be albe to have a look into the first lines of each file and "IDENTIFICATE" it by this way (with a little sophisticated database should it easily be Possible!!!)....

Like AMIGA showed us the future Tecnology in the past, as a system, that dies, because it was to to future for the present!

This is a real PETITION to Apple, to don't miss this train now, before the Hell of the H2Trillenium begins!!!!!!!!!!!

-&gt; REMEMBER: The Documents has to be the "State of the Art" unchanched of the different Filesystems, operating systems and mediums!!!.... The Documents are the real "CONTENT" of the FUTURE, not a OS with several differences to an other OS (if better or worser)....(why make it always as complicated, as it is?)

And "open Fileformat-Definitions" are the future! (not these undocumented Microsoft-like Fileformats, that lacks with the 3times larger Kilobytes than usual).... Sure, not Apple alone has to head into this direction, but if they don't, will Microsoft do? -&gt; no one will do it!...

This Open File-Format-Definitions would make it possible, to sell FileFormats once, to integrate in Customers Computers.... (that has 2 sides: � an economic forefait and an � "you never again have to convert something, after yuo have the Support of a Fileformat! -&gt; no programs has to be rewritten to support these and the others and again a new Fileformat, because the fileformats will be selled or released by it's Creators!... (if not, they get off the OPEN-Source Thougths and will be confrontated with their consequences sooner or later)!

;-)

[ 01-21-2002: Message edited by: tazmandevil ]

[cpt kangarooski]

I guess I'll ask: is that a joke, or is it Memorex?

Really, I don't believe anyone's been dismissive of using the concept of 'magic numbers' (which you're referring to -- in file metadata) as part of a larger strategy. However, it wouldn't be feasible alone.

IIRC, opening and reading a file to obtain relevant metadata -- something that would have to be done every time the file icon appeared -- is less efficient than storing that information in an external database.

Also of course, not all data formats are distinct. You can write a functional, if not proper, HTML file w/o tags that describe it as such. A computer would have a terribly difficult time working out that it was not a regular ASCII file. (or vice versa, that just because an ASCII file has "&lt;B&gt;" in it, this does not mean that it is HTML) Tab-delimited spreadsheet files are another example of this. God only knows what ASCII-art would be recognized as.

Furthermore, there are vast numbers of files and formats that exist today which would still have to be dealt with. Ignoring them is not a solution.

It also doesn't accomodate the ocassional need to change the file format descriptor (which is rare and unwise) or the more common need to change the program association.

Also of course, some file formats simply CANNOT accomodate metadata being placed into the file, because they were designed to expect that only the essential data would ever be present there.


No, probing the contents of a file to properly identify it is a good tool to keep in the toolbox of methods that the OS should use to assist the user. (generally only if the file is unidentified -- the found data should be plugged into the m.d. database, perhaps flagged as 'tenative') But it's not the best or first.

[MemeTransport]

Originally posted by cpt kangarooski:
<STRONG>
Also of course, some file formats simply CANNOT accomodate metadata being placed into the file, because they were designed to expect that only the essential data would ever be present there.


No, probing the contents of a file to properly identify it is a good tool to keep in the toolbox of methods that the OS should use to assist the user. (generally only if the file is unidentified -- the found data should be plugged into the m.d. database, perhaps flagged as 'tenative') But it's not the best or first.</STRONG>

I have a dream:

It's a dream that would probably make a mess out of cross-platform solutions but I have it anyway.

I'd like all files in OSX to be bundles in a similar way as app bundles (but ignoring all of the unix/down-in-the-basement heirarchy). Something like "foo.file". Inside the bundle would be (duh) the file and xml based metadata. The metadata structure should be flexible enough to allow for compound documents. It would be a god-send for desktop publishing among other things. All the resources needed plopped in together in one parsable file. The system could automatically wrap new files and provide basic metadata. Apps could extend the metadata following a standard DTD.

Metadata could include anything from creation date to a particular colorsync/icc profile. This alone would make apps much more interoperable.

[sadie]

MemeTransport was thinking:
<STRONG>I'd like all files in OSX to be bundles in a similar way as app bundles (but ignoring all of the unix/down-in-the-basement heirarchy). Something like "foo.file". Inside the bundle would be (duh) the file and xml based metadata. The metadata structure should be flexible enough to allow for compound documents. It would be a god-send for desktop publishing among other things. All the resources needed plopped in together in one parsable file. The system could automatically wrap new files and provide basic metadata. Apps could extend the metadata following a standard DTD.

Metadata could include anything from creation date to a particular colorsync/icc profile. This alone would make apps much more interoperable.</STRONG>

Interesting, but do you have any idea how slow that would be? Instead of pulling up a database entry for one file, it has to navigate into a folder and open a separate file from disk (which is noticably slower) then parse an XML format file (a lot slower) to find the folder's information - and you have to do this regularly.

For some file formats, it definitely makes sense to use a bundle. But not all of them - not a simple BBEdit text file or GIF image.

tazmandevil said something like:
<STRONG>Operating/File systems should not handle the filetypes anymore.... because filetypes are still available! Where? Yes, very simple:
IN THE FILES THEMSELVES!</STRONG>
(slighly cleaned up)

This makes some sense, but isn't enough. Many files do have a 'native' format, stated at the start of the file - including Java class files, XML files, and most complex document and media formats. However, there are quite a few formats that don't - because the program writers never thought it would be needed. If you rely only on the in-file-format, you'll get a lot of files that aren't identifiable.

File extensions, in-file-format reading and user input are all ways of discovering the metadata. Once it's been discovered, it should be held onto by the file system. When it needs to be transferred to another system, it can be downgraded to fit - file extensions added, bundles de-bundled, etc.

What this means is that Apple needs to be prepared to tear apart the Unix tools they got with BSD, and make them more intelligent when it comes to metadata. Darwin needs to become a real Mac-like command line, with the intelligence you'd expect from it, and not just another Unix clone.

[MemeTransport]

Originally posted by sadie:
<STRONG>
Interesting, but do you have any idea how slow that would be? Instead of pulling up a database entry for one file, it has to navigate into a folder and open a separate file from disk (which is noticably slower) then parse an XML format file (a lot slower) to find the folder's information - and you have to do this regularly.

For some file formats, it definitely makes sense to use a bundle. But not all of them - not a simple BBEdit text file or GIF image
</STRONG>

There would be more overhead. However, you are assuming that in-file metadata precludes a Be-like database. It doesn't. Adjustments to the database could be made by the system during normal saves. The database would not necessarily be XML but could be whatever (binary?) format seems best. Also, all the possible metadata would not be needed for the database...quite a bit of the potential information would only be of interest to applications and/or asset-tracking software.

The in-file metadata should be XML or some other industry standard. It makes sense to insure maximum parsability across applications and platforms. Apple could help ensure this by giving the Open Source folks suitable stub code for implementing the bundle/metadata system on their own platforms. The DTD should be public of course.

The file system database is really more of a propriatary thing. It's for managing an Apple file system. As long as developers have suitable API and docs, I think Apple could implement just about anything they want. You're right that speed is very important in this case.

Wrapping every file on the system is just about impossible. Wrapping all the low-level unix stuff would break everything. Wrapping files destined for other platforms or the web is definitely iffy. Some consistent way of dealing with this would be needed. After all, all pre-existing files wouldn't have the needed data.

Still, given the possibilities of extended metadata, I think files like BBedit's could be made much more useable with this system. Think revision tracking, user ID for changes made, source locations, comments on the file, sky's the limit. Creating a CVS system would get much easier. Files could carry much more information about themselves. You could have a single file that has many different language versions, or situation specific versions all in one file. You could include hinting for speech programs or machine translation. Best of all, none of it would be propriatary since the DTD is a public document. Sure, it would be fatty for some simple files but I think the gains make it worthwhile.

It's important for files to carry the information: if you give me a file do I get access to your file system's metadata-base?

Once a system like this is in place people will find new and novel ways to use it. It goes well beyond what we have today.

[sadie]

MemeTransport replied:
<STRONG>There would be more overhead. However, you are assuming that in-file metadata precludes a Be-like database.</STRONG>

Like i said, they can and should co-exist.

<STRONG>The database would not necessarily be XML but could be whatever (binary?) format seems best.</STRONG>

What is it with XML? Why does everybody seem to think it's a good format? It's as far from an efficient database as it's possible to get without shooting yourself.

<STRONG>The in-file metadata should be XML or some other industry standard. It makes sense to insure maximum parsability across applications and platforms. Apple could help ensure this by giving the Open Source folks suitable stub code for implementing the bundle/metadata system on their own platforms. The DTD should be public of course.</STRONG>

In-file metadata is specific to the format of the data, as is bundling. If you write a program, you can create your file formats using bundles, and you can write your in-file metadata in whatever format you see fit. But when formats are cross-platform there's nothing that Apple can do to force them to follow its own way of doing things.

Apple has to lead by example, not by law.

<STRONG>The file system database is really more of a propriatary thing. It's for managing an Apple file system. As long as developers have suitable API and docs, I think Apple could implement just about anything they want.</STRONG>

They could do the file-system metadata however they want, but BeOS demonstrated the enormous benefits of making the system open to application developers. HFS+ has the potential to support an identical or even better system than BeFS, if only it wasn't being strangled by being forced through an almost-standard Unix layer.

I think we can all agree that Apple's current approach, not bothering with file-system metadata, is the worst of all worlds.

[footar]

Originally posted by absmiths:
<STRONG>Oh please! No XML! XML is such an overused data format. What is it? Merely a heirarchical data structure stored in the worst possible format (usually Unicode text) requiring the slowest possible interpretation (parsing). Imagine a basic metadata for a simple file:

Immutable bit set
Creator 'AdOb'
Type 'File'

Binary [9 bytes, not 5, Apple uses 32 bits for type/creator, right?]:
FF AdOb File

XML [74 bytes, 148 if unicode!]:
&lt;Immutable&gt;yes&lt;/Immutable&gt;
&lt;Creator&gt;AdOb&lt;/Creator&gt;
&lt;Type&gt;File&lt;/Type&gt;

I for one don't care to increase the storage of metadata by 7-15 times just to make it human readable!</STRONG>

First off, representing 74 letters in Unicode could take 74 bytes if you used UTF-8, 148 for UTF-16 or 296 for UTF-32. Don't spout off about Unicode until you've read the specs, or at least know what a transformation is. Second, the fact is that many Mac users don't speak English.

Third, XML is a language in the same sense C is a language. Just as we can compile C to binary and optimize it, XML can be represented in binary, even though it doesn't need to be compiled. This is how all XML databases work.

<STRONG>I work for a company that has done just that - bought into the XML craze and implemented an XML middleware component which does XML messaging. It sucks, is slow, and makes no more sense to the receiving end than a published binary structure.</STRONG>

Marshalling and unmarshalling an XML message in text form is not inherently faster or slower than any equivalent binary format. The key is equivalent: binary formats are generally hard to extend, unless you go in for recursive graph structures. When that happens, you're doing about as much work as you would with XML.

Furthermore, the fact that nearly all the dominant Internet protocols use a simple ASCII based protocol to handle their control structures (Telnet, FTP, HTTP, MIME, etc, etc) is testament to the fact that human readability is crucial to the development of a standard protocol. Actually getting your protocol used is far more important than a few clock cycles.

<STRONG>Besides, human readability is not a valid requirement for this type of system since it is only useful during debugging/trouble shooting - and a tool to read the data for the user would be written in any case.</STRONG>

Yeah, who needs debugging and troubleshooting?

[Millennium]

As a rather heavy user of XML, I'm somewhat startled by the XML-bashing that's going on here.

What is it with XML? Why does everybody seem to think it's a good format? It's as far from an efficient database as it's possible to get without shooting yourself.

I don't understand this. What's so inefficient about it?

The real problem with XML is that it gets misused for a lot of things. People seem to get this scatterbrained idea that XML makes a good database. It doesn't, and it was never meant to even be a database.

XML is meant for applications where performance is not critical. People complain about the "inefficiency" of XML; these people are completely missing the point. Performance doesn't matter when you're reading in, say, a configuration file (done relatively rarely) or a Word document (where it probably takes longer to format the document for onscreen use than it did to read and parse the thing in the first place). Even when it comes to reading Web pages, reading in the file isn't a big deal on the Web server (which doesn't even have to parse the file), and XML is more efficient than HTML was to parse. And these are the kinds of things XML was meant for. Not for storing databases.

It also makes a great intermediate format for Web programs moving to database access. XML isn't the fastest thing out there, but it's better than regex'ing your way through a semantically-meaningless HTML file, particularly when you consider table-based design and the utter mess it makes of HTML code. Clean HTML code is nicer to work with, but it's still not great. I've been trying to port WWWBoard over to an XML-based file format as an exercise. It's proven quite enlightening; I'm not done with it yet, but my performance is already better, and my data file sizes are much smaller. Not to mention that I've fixed the Y2K-compliance issues and all the security holes I could find (of which there were many).

For my own messageboard script (the one I'm developing to actually use on my own site), I'll probably end up using some kind of database. But the XML-usage port has been a great learning experience, and I think I'm really starting to get a handle on what XML is supposed to be about.

Oh, and one final note, to those who say it's inefficient and slow: what API are you using?

[Richard Less]

(Hmm... I was logged in as footar before, wonder what happened? Oh well.)

Originally posted by Millennium:
<STRONG>As a rather heavy user of XML, I'm somewhat startled by the XML-bashing that's going on here.

I don't understand this. What's so inefficient about it?

The real problem with XML is that it gets misused for a lot of things. People seem to get this scatterbrained idea that XML makes a good database. It doesn't, and it was never meant to even be a database.

XML is meant for applications where performance is not critical.</STRONG>

Yup. Like many Internet protocols, XML is a lightweight version of another protocol. (SGML, in case you were wondering, which was the standard that was used to describe HTML. XML can also be described using SGML, and incidentally, XML can *not* describe HTML.) It can *really* only be used to serialize and deserialize, which means you generally process the whole thing in one go, and that heavily limits what you can do. It doesn't adhere to a relational model, so it's useless for a proper database, even as a conceptual system. That's because there is no concept of constraints.

What is it good for? Simple: it's Unicode + attributes. Unix has survived for decades subsisting on flat ASCII files for its system administration, and XML can do all that and more. No more tab delimited files, just put it in some kind of table tags. All human readable, safe to go through email, no line-ending corruption, etc. When you just need a place for everyone to stash some persistent data, it seems a hierarchy is the way to go. This is the model that has been used by filesystems and things like the Window's registry or Mac OS X's new NetInfo.

For a thorough treatment of XML, you should read Fabian Pascal's Against the Grain series. His homepage is at Database Debunkings and is required reading for any serious DBA or software developer.

He's a tough read, but well worth it.