[Big Mac]

Where were was the metadata equivalent to the OS X metadata files stored in OS 9? It wasn't the Desktop Database. Where was it? Finder Preferences? The only invisible files in regular folders on OS 9 were icon files pasted on to folders and the like.

Also, why couldn't the metadata files be stored somewhere in ~/Library/? Why is it necessary to store them in the folders individually?

[besson3c]

Originally Posted by Big Mac 

Where were was the metadata equivalent to the OS X metadata files stored in OS 9? It wasn't the Desktop Database. Where was it? Finder Preferences? The only invisible files in regular folders on OS 9 were icon files pasted on to folders and the like.

Also, why couldn't the metadata files be stored somewhere in ~/Library/? Why is it necessary to store them in the folders individually?

Maybe the .AppleDouble and .AppleSingle directories?

[Big Mac]

Originally Posted by besson3c 

Maybe the .AppleDouble and .AppleSingle directories?

Oh yeah. Now I know - it had to have been the resource fork, and .AppleDouble and .AppleSingle were the replacements for the resource fork.

[Chuckit]

Originally Posted by besson3c 

Because it is not transparent in having these other side-effects. There are surprises, some which could literally be detrimental (e.g. my md5 hash example).

Shouldn't a good GUI produce predictable results that make life easier than doing the same operation via the command line?

To be honest, your MD5 hash example isn't completely invalid, but it strikes me as rather contrived and simply an example of using the wrong tool for the job. Finder's Archive command is a way of quickly and reliably creating an archive of files on Mac OS X. Complaining that it could mess with the MD5 in some situations is like complaining that two textually identical UTF-8 files might have different MD5s depending on whether the editor was using composed characters.

More relevant: I just compressed a copy of my Fonts folder with the command-line zip tool. When I decompressed it, my fonts were ruined. I did the same with the Finder's Archive command — the MACOSX folder saved the day!

Again, the MACOSX folder does not render the zipfile unusable in any way. If somebody has a conniption fit because there's a folder they're not interested in using, they need to call the WAAAHmbulance.

[Big Mac]

Originally Posted by Big Mac 

Oh yeah. Now I know - it had to have been the resource fork, and .AppleDouble and .AppleSingle were the replacements for the resource fork.

However, thinking about it some more, only some files had resource forks. AppleSingle and AppleDouble were the resource fork replacements I think, but that still does not account for .DS_Store and its equivalent in OS 9.

[besson3c]

The Finder/GUI tool creates zip files, so the fact that it behaves differently than other zip tools and does not provide you with any feedback or indication of this, nor a way to configure this creates problems. Obfuscation is not a good thing, and users shouldn't have to read threads like this to learn that these extra files are created.

In many ways, it seems like it would be better design to *not* hide these directories.... Just thinking outloud here.

[Chuckit]

Originally Posted by besson3c 

The Finder/GUI tool creates zip files, so the fact that it behaves differently than other zip tools and does not provide you with any feedback or indication of this, nor a way to configure this creates problems. Obfuscation is not a good thing, and users shouldn't have to read threads like this to learn that these extra files are created.

I wouldn't call this "obfuscation." It deals with data that most other zip tools ignore — and I don't know about you, but I consider data loss to be unacceptable — and handles it in a simple and 100% compatible way. Users shouldn't have to be bothered with details of how the files are stored as long as it doesn't affect data integrity. If somebody wants to know, it's answered with a very simple search.

Originally Posted by besson3c 

In many ways, it seems like it would be better design to *not* hide these directories.... Just thinking outloud here.

The directory isn't hidden. If the zip is unarchived by the Finder, it reconstructs the resource fork. If the zip is unarchived by a tool that doesn't correctly handle resource forks, it unarchives a MACOSX folder that preserves the data.

[Big Mac]

So does anyone have a reason why the metadata files could not be stored in a central place in ~/Library/?

[Chuckit]

No. I assume it was done for speed (the same reason .DS_Store is in some unreadable binary format), but the entire idea doesn't seem to have been thought out that well.

[Big Mac]

Originally Posted by Chuckit 

No. I assume it was done for speed (the same reason .DS_Store is in some unreadable binary format), but the entire idea doesn't seem to have been thought out that well.

Yeah, I thought about speed concerns. OS X's designers thought it was a superior choice to do it this way for compatibility with the outside world rather than the way the classic OS did it. OS X's solution does negate the use of the MacBinary format, so I guess there's a tradeoff either way. There should always have been a Finder option not to create the metadata files on foreign drives.

[Chuckit]

In case you haven't read it, the guy behind .DS_Store write an interesting blog entry on the subject.

The comments contain interesting tidbits as well, such as this:

At the time, there were two camps at Apple: those who believed that pervasive metadata could lead to a better user experience and those who thought that constraining yourself to the capabilities of the Unix file system would make interoperability with other OSs easier.

Thankfully, the value of metadata is now finally recognized and Mac OS has some of the best metadata support, but we still seem to be stuck with decisions from seven years ago.

[peeb]

As well as this:

"There is also an unfortunate bug that is not fixed to this day that result in an excessive creation of .DS_Store file. Those files should only be created if the user actually makes adjustments to the view settings or set a manual location for icons in a folder. That's unfortunately not what happens and visiting a folder pretty much guarantees that a .DS_Store file will get created"

Which is the bug we're talking about!

[peeb]

And, as another user commented: "The whole concept of storing view settings with the file system node is a bad idea on a multiuser system, or a system that logs onto multiuser systems. What you really want is to capture the user's view settings somewhere in ~/Library. That keeps them stable for network users, keeps them from polluting network server directories"

[Hal Itosis]

So . . . Windows renders dot files as visible?
What is the purpose of "looking" at dot files?
Is there no option in Windows to hide dot files?
Wow . . . that does sound annoying.

[Chuckit]

Originally Posted by Hal Itosis 

So . . . Windows renders dot files as visible?
What is the purpose of "looking" at dot files?
Is there no option in Windows to hide dot files?
Wow . . . that does sound annoying.

That's like asking, "So…Mac OS X renders thumbs.db files as visible? What is the purpose of looking at thumbs.db files?"

Dot-files being invisible is a Unix convention. Mac OS 9 and Windows are not Unix-like operating systems, so they show dot-files. Mac OS X is based on BSD, so it hides them.

[Hal Itosis]

Can't we all just get along?

Seriously, I already knew all that.
Any OS can hide or show anything it WANTS to.
OS9 (which is irrelevant btw) could have been tweaked if Steve *wanted* to.
Microsoft can be more accommodating to the rest of the world, if they WANT to be.

This entire "argument" (if that's what it's supposed to be) is silly.

I can't figure out why QuickBooks (and MYOB, etc.) have different formats
for their data files. Why must we convert/import/export QuickBooks files
going from one OS to another? It's a freaking QuickBooks file. A data structure!
And then it doesn't turn out to be totally compatible if it's edited and returned!

Now **that's** annoying. (Not these silly dot files). Get a life people.

[KP*]

Originally Posted by peeb 

As well as this:

"There is also an unfortunate bug that is not fixed to this day that result in an excessive creation of .DS_Store file. Those files should only be created if the user actually makes adjustments to the view settings or set a manual location for icons in a folder. That's unfortunately not what happens and visiting a folder pretty much guarantees that a .DS_Store file will get created"

Which is the bug we're talking about!

Very interesting to see that this is considered a bug and not a feature. I wouldn't mind so much the creation of .DS_Store files only in the folders I customized. I hope someday they get around to fixing it.

[Big Mac]

Originally Posted by peeb 

And, as another user commented: "The whole concept of storing view settings with the file system node is a bad idea on a multiuser system, or a system that logs onto multiuser systems. What you really want is to capture the user's view settings somewhere in ~/Library. That keeps them stable for network users, keeps them from polluting network server directories"

Which is what I was advocating. It's the sensible way to manage the files. But the question is, is it fast enough?

Originally Posted by Hal Itosis 

Can't we all just get along?
Seriously, I already knew all that.
Any OS can hide or show anything it WANTS to.
OS9 (which is irrelevant btw) could have been tweaked if Steve *wanted* to.
Microsoft can be more accommodating to the rest of the world, if they WANT to be.

I only brought up OS 9 because it had a superior scheme for managing .DS_Store class metadata.

This entire "argument" (if that's what it's supposed to be) is silly.

The proliferation of the files is annoying to some, especially when they're dealing with foreign filesystems.

I can't figure out why QuickBooks (and MYOB, etc.) have different formats
for their data files. Why must we convert/import/export QuickBooks files
going from one OS to another? It's a freaking QuickBooks file. A data structure!
And then it doesn't turn out to be totally compatible if it's edited and returned!

Now **that's** annoying. (Not these silly dot files). Get a life people.

When I first figured that out about QuickBooks I was terrifically annoyed as well. I cannot comprehend why the files aren't cross platform. But that's a third party problem. This is a core OS issue that is problematic for some, so there's no reason to dismiss it in the perfunctory fashion you have.

[PhotoMacUser]

I had a zipped folder get kicked back as SPAM/Virus infected because it had ".DS" and "__MACOSX" in the zipped folder. I next experimented using drop stuff and zipping files in and out of a folder, next I used "create archive" in OSX. I opened the zipped files in my Virtual PC using Windows 2000 and all the extra stuff was in there-Not good! I did some searching on Versiontracker and I found BetterZip.app. It creates zips without the extra MAC OSX stuff in it. I highly recommend it for those exchanging files regularly with PC users and do not want to confuse the issue. The other files that often show up are the thumbnails that accompany image files. I cannot believe that Stuffit-Dropstuff does not give you the option to remove the extraneous files to prepare for PC users.

[Hal Itosis]

Originally Posted by Big Mac 

The proliferation of the files is annoying to some, especially when they're dealing with foreign filesystems.

If Windows littered my Mac with files or something, I might write an
AppleScript/Automater action... or some shell script... and kick-start
them with a Folder Action, or perhaps cron/launchd. (I think someone
already mentioned a similar idea, from the Windows POV). So, surely
Windows users have similar tools to deal with simple issues like this.

Originally Posted by Hal Itosis 

What is the purpose of "looking" at dot files?
Is there no option in Windows to hide dot files?

I don't think I got *satisfactory* replies to those questions,
so I can't be of much further help here I guess... so sorry.

--

I think the QuickBooks problem is the same as the OS problem:
neither party wants to take enough time to make our computing
life any easier in cross-platform contexts. (greed, laziness, etc.)

[Chuckit]

Originally Posted by Hal Itosis 

I don't think I got *satisfactory* replies to those questions,
so I can't be of much further help here I guess... so sorry.

What's unsatisfactory about, "No, there is no option to hide dot-files in Windows because they have no special meaning in Windows, so it would be somewhat like an option to hide files beginning with the word 'bacon'"?

[peeb]

Let's just hope it gets fixed in Leopard. It's the old cry of FTFF!

[kmkkid]

Originally Posted by peeb 

The thing is, they don't leave traces of that garbage on everything they look at... It's all very well to say that 'they need to get over it', but, in my case, they are doing me a favor by giving me access to their drives, and are annoyed that I leave junk they don't want all over them. Plus, the registry does not store folder preferences.

No the registry doesn't hold all folder preferences, those stupid thumb database and desktop.ini files do, which I might add are visible in OS X and littered all over. It's a no-win situation, because both OS' have different methods of making files invisible.

[Hal Itosis]

Originally Posted by Chuckit 

What's unsatisfactory about, "No, there is no option to hide dot-files in Windows because they have no special meaning in Windows, so it would be somewhat like an option to hide files beginning with the word 'bacon'"?

Thanks, that's better than before.

Well, OSX can do something like this:

COUNT ALL .DS_STORE FILES:
locate .DS_Store | wc -l

ERASE ALL .DS_STORE FILES:
locate .DS_Store | tr '\n' '\000' | xargs -0 rm -f


If Windows can so efficiently convey viruses, worms, etc., on behalf of hackers...
surely it can pull off some similar tricks as shown above for the benefit of users.

--

I agree OSX shouldn't copy those items across network shares...
but -- then we OSX can also prevent that by cleaning what we
transfer. I've seen OSX applications that contain a bunch of junk
INSIDE the apps... even Sherlock .FBC files. They should clean out
such stuff before they make their dmgs.

[peeb]

Originally Posted by kmkkid 

No the registry doesn't hold all folder preferences, those stupid thumb database and desktop.ini files do, which I might add are visible in OS X and littered all over. It's a no-win situation, because both OS' have different methods of making files invisible.

My point is not that Windows handles it better, but simply that OSX doesn't handle it well. We should hold ourselves to a higher standard. As noted above, in principle, window preferences should be stored locally on multi-user drives.

[besson3c]

Originally Posted by kmkkid 

No the registry doesn't hold all folder preferences, those stupid thumb database and desktop.ini files do, which I might add are visible in OS X and littered all over. It's a no-win situation, because both OS' have different methods of making files invisible.

All the more reason why we need a metadata standard at the file system level, but unfortunately "standard" and Microsoft are usually very much oil and water.

[peeb]

Well yes, but this is not file meta-data we're talking about, it's user directory preferences.

[Chuckit]

Originally Posted by peeb 

Well yes, but this is not file meta-data we're talking about, it's user directory preferences.

It's directory metadata.

[peeb]

Originally Posted by Chuckit 

It's directory metadata.

Kind of, it's data about a particular user's preferences on how the directory should be displayed, not an inherent property of the directory. It should be stored in the user's preferences, not the directory.

[Chuckit]

Originally Posted by peeb 

Kind of, it's data about a particular user's preferences on how the directory should be displayed, not an inherent property of the directory.

Ditto for labels, comments, permissions, resolution settings on photos (doesn't change the pixel data) — metadata is just data about data. Preferences directly relating to a particular file are metadata for that file.

[peeb]

Well, it depends whether they are meta data about the file or directory itself, or about a particular user's preferences about them. On a multi-user system, you don't want each users' preferences stored in that file or directory. Fine if it is the time the photo was taken, but not if it is the time-zone of the user viewing it. Fine if it is the resolution of the file, not if it is related to the resolution settings of a particular user's monitor setup.

[miawj]

Sorry to interrupt the philosophical discussion but is there a practical solution to copying nested folders back on to a Macintosh? Although I can browse CDs of Macintosh files burned by Windows machines I can not copy them back to a Macintosh without hanging on the unhidden hidden files.

[peeb]

Originally Posted by miawj 

Sorry to interrupt the philosophical discussion but is there a practical solution?

You must be new here

[Scott-G]

Originally Posted by Big Mac 

So does anyone have a reason why the metadata files could not be stored in a central place in ~/Library/?

I think the .ds files should stay where they are. This way when you mount a portable drive, for example, the directory view information will move with the drive from one machine to the next. And if you never mount it again you will not have unused metadata cluttering your ~/Library folder.

[besson3c]

Associating folders with metadata might be a little difficult if they were placed in ~/Library. I'd imagine the most logical way to do this would be to use the inode of the folder as a unique ID to associate the folder with the metadata, but what happens when that inode is reused? You can't really use the name of the folder, because what happens if there are two folders in separate locations with identical names?

Anohter problem: if you move this folder to another computer, and then back to the Mac, what happens to the metadata?

The best way to handle this would probably be to do maintain this data at the file system level. This is exactly what xattr is for. People are inclined to throw data into a database, and of course databases have their usage, but this seems more appropriate for a filesystem to me.

[peeb]

Originally Posted by Scott-G 

If you never mount it again you will not have unused metadata cluttering your ~/Library folder.

Right. Much better to have it cluttering up every folder you ever look at on someone elses drive...

[Scott-G]

Originally Posted by besson3c 

The best way to handle this would probably be to do maintain this data at the file system level. This is exactly what xattr is for. People are inclined to throw data into a database, and of course databases have their usage, but this seems more appropriate for a filesystem to me.

So where does xattr data getted stored at the file system level? Or is that what the .ds_store files are?

[Scott-G]

Originally Posted by peeb 

Right. Much better to have it cluttering up every folder you ever look at on someone elses drive...

Right, because when that directory is deleted it is gone. Not left in 20 different users ~/Library folders until they do a clean install.

[besson3c]

Originally Posted by Scott-G 

So where does xattr data getted stored at the file system level? Or is that what the .ds_store files are?

File system level meaning literally that - not file structure, but actually part of the internal organization of the file system. Xattr stores extended attributes about files that, AFAIK, is attached to inodes within the data tree. My implementation details are very sketchy, but you can learn more about this within the Wikipedia.

[besson3c]

Originally Posted by Scott-G 

Right, because when that directory is deleted it is gone. Not left in 20 different users ~/Library folders until they do a clean install.

The delete commands could trigger deletes in ~/Library, but this would require modifying the BSD rm command, and any other delete mechanism that doesn't piggyback on rm.

[Scott-G]

Originally Posted by besson3c 

The delete commands could trigger deletes in ~/Library, but this would require modifying the BSD rm command, and any other delete mechanism that doesn't piggyback on rm.

And this would only remove it on the machine that did the rm. Not any other machines that may have mounted the volume in the past.

[besson3c]

Originally Posted by Scott-G 

And this would only remove it on the machine that did the rm. Not any other machines that may have mounted the volume in the past.

True...


These cross-platform issues are probably another reason why a more file system centric approach is better than a database centric approach. If you were to copy a file off the Mac and then back to the Mac so that a copy of the file is created, assigned a new inode, I'm not sure how you could associate this metadata with a DB entry.

The data traveling with the file seems like the way to go, which xattr will do. I would imagine that the only way to stop the littering is to make an RFC out of xattr, if there isn't one already, and pressure Microsoft to support this standard.

[Chuckit]

That hasn't worked with HTML.

[besson3c]

Originally Posted by Chuckit 

That hasn't worked with HTML.

No, but HTML support is also sort of a relative thing - not all browsers support every facet of the CSS 3 spec flawlessly, for instance.

Xattr should hopefully more or less be a "support it" or "don't support it" sort of thing, once the spec is mature enough to not be a moving target.

[gberz3]

Hi All,

I'm looking to solve this issue with ".DS_Store" and "._filename" files created on Network drives using OS 10.4.10. Basically the old faithful "defaults write com.apple.desktopservices DSDontWriteNetworkStores true" works, but only for ".DS_Store" files. The "._filename" files still get written. I've also tried BlueHarvest but it doesn't seem to have any effect on the "._filename" files either.

So I have two questions:

1) Is there a way, perhaps through Growl, to get system notifications on file copies and then check for a "._filename" to delete?

2) Does anyone have an *automated* solution that actually works for both?

Thanks!

[OreoCookie]

There's no need to revive two threads on the same topic and pasting identical replies into each of them.

[gberz3]

Understood, but I did it for the same reason you posted your message in both threads; because not everyone follows every thread. I'm hoping to get as wide an audience as possible without wreaking havoc. It just makes sense. Besides, isn't this method preferable to creating yet another thread regarding ".DS_Store"?

That said, is anyone aware of a solution?

[OreoCookie]

There's no point to resurrect two threads, one suffices. It won't widen your audience either, it just aggravates people who might be able to help.

[gberz3]

Listen, OreoCookie. I respect your position. I really do. Not to mention the fact that you're a mod so there's no point in truly "arguing". But I'll explain to you how many people in these forums work. Not everyone peruses threads all day. And not everyone is part of every thread. Many folks, like myself, only check threads on the weekends, or when I get a nice email saying "Someone has replied to your thread". With different people being subscribed to different threads, one of those people might very well have come upon a solution and not posted. That said, when they get their nice little email saying "Someone has replied to your thread", they might say "oh, right, I found that a while ago, here's the solution".

I found several *pages* worth of related threads. I posted to only two. I'd say that's reasonable. It's not that serious, but it *is* that simple. . .

[MacosNerd]

Originally Posted by gberz3 

Listen, OreoCookie. I respect your position. I really do. Not to mention the fact that you're a mod so there's no point in truly "arguing". But I'll explain to you how many people in these forums work.

Wow a noob telling a mod how this forum works

I think Oreo has a better grasp on how Macnn works then you do and he has a very valid point. You don't wish to argue with him yet you do and tell him Macnn works.