Welcome to the MacNN Forums.

CharlesS · Jun 19, 2002, 12:15 AM

First of all a disclaimer. I did this purely for scientific reasons. I'm posting this in case someone else may find it interesting. I am not recommending that you do this on your production system! If you decide to perform this hack, do it on a backup system that is on an external disk or second partition.

Anyway:

I'm posting this topic from a machine that has all the required UNIX files and folders in /System instead of on the root of my disk.

How is this possible, you ask? The answer is quite simple - while booted from another hard drive, I copied the following files inside the System directory: bin, dev, private, sbin, and usr. Then, I made symbolic links to the folders inside System, and put them on the root of the disk. A bunch of other folders and files, like .vol, .Trashes, Volumes, automount Desktop DB, and Desktop DF, are automatically generated and thus can be deleted. This leaves the mach files, and therein lies the problem. The reason this didn't work the last time I tried this several months ago was that although most of the folders are fine with just having a symbolic link at the root of the drive, the mach_kernel file wants to be at the root of the drive, or it won't work. There is a solution, however. First, you can delete the mach and mach.sym files, as they are automatically generated at startup. Now, although mach_kernel needs to be at the root of the disk, since it is a file it can still be stored inside System by making a hard link (use the ln command without the -s flag). The .hidden file can also be hard-linked, since it is a file. This is what I did, because I didn't want to test it with a soft link. For all I know, the .hidden file may work if only a soft link is present at the root of the drive - this is one thing I have not tested.

But to make a long story short, if you install OS X on a secondary partition, move the UNIX folders inside System, delete the files and folders that are automatically generated, and set up the links, you will end up with practically nothing but the System, Applications, Library, Network, Users, and your invisible links, and you will nonetheless have a bootable system that will work just fine, including being compatible with UNIX apps! In addition, you have the benefit of being able to browse the UNIX folders with the Finder, by going through System.

Now the reason you shouldn't do this on your production system is that Installer.app still has an issue with symbolic links in install paths, meaning that once you perform this modification, many package installers that install UNIX files or OS updates will no longer install properly. However, the upshot of this is that it would be fairly easy for Apple to distribute the system in this way, and have it possible to transplant a system and all its UNIX folders simply by dragging the System directory to another drive/partition. The Finder could ask for your admin password, and then copy the folder and automatically set up the appropriate links on the drive and bless the folder. Presto, you'd have the Classic Mac OS ease of moving the system, without the need for a utility like Carbon Copy Cloner. The only other thing they'd have to do would be to fix the Installer, which they should be able to do since I was able to make Pacifist respect symbolic links.

Yes, this is a somewhat obvious hack, but AFAIK no one had ever actually tried it before, and it has rather large implications IMHO, so I decided to be the one to try it...

[ 06-19-2002, 12:19 AM: Message edited by: CharlesS ]

Big Mac · Jun 19, 2002, 12:40 AM

Would be very nice, Charles. Thanks for testing it out - hopefully you've sent Apple feedback on the subject.

booboo · Jun 19, 2002, 09:55 PM

I can't believe this post had so few replies...!

CharlesS mod is really important, and definitely the way OS X should go. The ease of creating bootable drives by simply dragging 1 folder always empowered Mac users in the past, and always exposed the pain of windows in this respect...

CharlesS, please keep us posted, and email apple too...

Thanks :-)

raviruddarraju · Jun 19, 2002, 10:10 PM

Man, this is certainly a cool thing. I am expecting Apple to do something like this before. I mean, what the heck, Apple's all about ease of use!! But, awesome work.

- Ravi

Drizzt · Jun 19, 2002, 10:23 PM

The reality is different.. I don't want to burst you ballon.. but you did almost nothing..

For MacOS 9 speakers.. it would be like this :

You move your Documents folder in the 'System Folder' and put an alias to Documents on the root of your hard drive.

The system acts like before, but your files are in 'System Folder'.

The Unix core understands links correctly, that's why everything works fine.. but the Cocoa API still struggles it seams..

When I read the title, I thought : "Wow.. someone changed all the environnement variable to move stuff in the /System folder". That would require more work, but if all the variables are changed (including those used in Cocoa and Carbon), the system wouldn't see anything at all and work correctly <img border="0" title="" alt="[Wink]" src="wink.gif" />

Sorry if it looks like a rant or something.. it isn't.. and I don't want to leave a bad after-taste neither <img border="0" title="" alt="[Wink]" src="wink.gif" />

Millennium · Jun 19, 2002, 11:12 PM

While it's an interesting trick, it's not all it's cracked up to be. Did you actually test this by dragging your /System folder to another partition?

For starters, you missed the /etc folder, and I know you can't simply symlink that one out or the system won't boot.

As for the hardlinks you did for files, I'm afraid your method isn't as foolproof as you might think. What hardlinking does is put the file in multiple places on the same disk. This is not the same as copying the file, because there is still only one file; it's just in multiple places at once (don't try visualizing this at home, kids; any real-world metaphor would violate several major laws of physics. Computers are just Cool Like That). So when you copied the files into /System and then hardlinked them back out, it wasn't necessary: you could have simply not moved the files and hardlinked them into /System; the end result would have been functionally identical. Incidentally, this is why the Unix system call to delete a file is called unlink; it really just removes the hardlink to a file that you mention, and if there are no hardlinks left for that particular file only then does it truly delete the file. This makes hardlinks kind of neat, because no matter which one you delete, you will never orphan the other hardlinks (like you can do with aliases if you delete the original, rather than simply moving or replacing it). Unfortunately, hardlinks only work on files, not folders, and you cannot make a hardlink on one volume to a file on another. Otherwise, they'd beat the pants off of even aliases.

Interesting -even fine- work. But I'm afraid that while you've come close to the mark, you have not hit it.

cwasko · Jun 19, 2002, 11:53 PM

Well, in order to 'hit it' would require mass support from Apple. I would love to see *everything* moved to inside the /System directory. Even changing the bootup process to look in /System for the kernel would be great. however, I'm not sure this part is even possible. Then, at boot up, the kernel could just make soft links at the root to all the UNIXy type of directories in /System. This would be awsome becasue all standard UNIXy apps would work (I think) and then we'd have the ability to ditto the entire /System directory and maintain the usefullness of the System.

Actualy, the more I think about it, /usr/local should really reside in /Library. I believe the intent for these 2 directores was the same (back in the NeXT and Rhapsody days of /Local). Meaning, the user should never have to touch /System and all user installed files should go in /Library. Just like the user should never have to touch anything in / but all user installed files should go in /usr/local. The benfits of this are tremendous as you can do a clean install of your system and maintain *all* your custom installations. Anyhow... back to reality.

Mithras · Jun 20, 2002, 12:21 AM

I also had idly considered this but never followed through; so I give props to CharlesS for doing it.

I like cwasko's idea of /usr/local being in Library. Speaking of which, why do kernel extensions install into System? Shouldn't there be a /Library/Kernel Extensions/ folder too?

That said, the benefits of this setup will necessarily be relatively limited. You'll still need to run an installer or use a command-line tool (or GUI wrapper; but not the Finder) to copy a /System from one disk to another. That's because you need to preserve owner, group, permission, setuid bit, and so on, and a simple Finder copy will wreck all of those.

CharlesS · Jun 20, 2002, 03:12 AM

<blockquote>quote:<hr />Originally posted by Drizzt:
The reality is different.. I don't want to burst you ballon.. but you did almost nothing..<hr /></blockquote>Yeah, specifically what I did was to test a simple hack that several have mentioned earlier but no one has actually tried. I verified that it works. I'm not claiming to have rewritten the kernel or anything... just proven that Apple could do this without too much trouble if they wanted to. The fact that this was a simple operation was actually part of the point.

<blockquote>quote:<hr />For MacOS 9 speakers.. it would be like this :

You move your Documents folder in the 'System Folder' and put an alias to Documents on the root of your hard drive.

The system acts like before, but your files are in 'System Folder'.<hr /></blockquote>No, in Mac OS 9 terms, it would be more like the System files as well as various extensions and support files were littered at the root level of the drive and made invisible, and I moved them into the System Folder.

<blockquote>quote:<hr />The Unix core understands links correctly, that's why everything works fine.. but the Cocoa API still struggles it seams..<hr /></blockquote>In what way?

CharlesS · Jun 20, 2002, 03:21 AM

<blockquote>quote:<hr />Originally posted by Millennium:
While it's an interesting trick, it's not all it's cracked up to be. Did you actually test this by dragging your /System folder to another partition?<hr /></blockquote>No, but I prepared the system such that the only things that were there were the things that would be present if the user had dragged System, Library, Users, and the other visible folders, and the system had automatically set up the symlinks.

<blockquote>quote:<hr />For starters, you missed the /etc folder, and I know you can't simply symlink that one out or the system won't boot.<hr /></blockquote>Millenium... type "ls -al /" in the Terminal once, look at etc, and tell me what you see... it's a symlink. Yes, that's right, the etc folder is not at the root level at all - it's in /private. So not only is it true that you can symlink it out, but Apple does symlink it out. I didn't have to create that symlink because one already existed that pointed to /private/etc, and I moved /private and left a symlink in its place so the existing symlink still worked. If the Finder could automatically create the symlinks when dragging a system, it would make the symlinks for etc (and also for tmp, cores, and var). As these symlinks already existed, it's unnecessary for the purposes of this demonstration.

<blockquote>quote:<hr />As for the hardlinks you did for files, I'm afraid your method isn't as foolproof as you might think. What hardlinking does is put the file in multiple places on the same disk. This is not the same as copying the file, because there is still only one file; it's just in multiple places at once (don't try visualizing this at home, kids; any real-world metaphor would violate several major laws of physics. Computers are just Cool Like That). So when you copied the files into /System and then hardlinked them back out, it wasn't necessary: you could have simply not moved the files and hardlinked them into /System; the end result would have been functionally identical. Incidentally, this is why the Unix system call to delete a file is called unlink; it really just removes the hardlink to a file that you mention, and if there are no hardlinks left for that particular file only then does it truly delete the file. This makes hardlinks kind of neat, because no matter which one you delete, you will never orphan the other hardlinks (like you can do with aliases if you delete the original, rather than simply moving or replacing it). Unfortunately, hardlinks only work on files, not folders, and you cannot make a hardlink on one volume to a file on another. Otherwise, they'd beat the pants off of even aliases.<hr /></blockquote>Yes, I know how hardlinks work, thanks. I actually made a hardlink from the file in /System because I first tried softlinking the file and it didn't work for mach_kernel, and at this point the file was already copied inside /System. For .hidden, I actually did just hard link it from its original location.

As for working across volumes - it doesn't need to! When you drag /System to another volume under my theoretical system, it would copy the contents of /System, including mach_kernel. The Finder could then automatically set up a hard link to the root of the drive when it created the symlinks to the folders. And as for only working with files, not folders - yeah, that's why I used it for the items that were files - mach_kernel and .hidden. I even mentioned this in my original post. Fortunately, soft links seem to be good enough for all the folders.

<blockquote>quote:<hr />Interesting -even fine- work. But I'm afraid that while you've come close to the mark, you have not hit it.<hr /></blockquote>What mark? I'm not pretending to have rewritten the kernel or solved world hunger or anything. All I did was prove that something is possible. The only one that can "hit it" is Apple...

CharlesS · Jun 20, 2002, 03:26 AM

<blockquote>quote:<hr />Originally posted by Mithras:
I also had idly considered this but never followed through; so I give props to CharlesS for doing it.

I like cwasko's idea of /usr/local being in Library. Speaking of which, why do kernel extensions install into System? Shouldn't there be a /Library/Kernel Extensions/ folder too?

That said, the benefits of this setup will necessarily be relatively limited. You'll still need to run an installer or use a command-line tool (or GUI wrapper; but not the Finder) to copy a /System from one disk to another. That's because you need to preserve owner, group, permission, setuid bit, and so on, and a simple Finder copy will wreck all of those.<hr /></blockquote>Yeah, that's true. I'm thinking of a future, turbocharged Finder that can have an option for admin users where you can authenticate before copying files, causing them to be copied by root, and therefore preserving the permissions, etc. That's why I mentioned that the Finder would ask you for your admin password when you tried to do this.

The Finder would need root access to make the symlinks and have them owned by root like they should be anyway, so it's kind of a given for my scenario.

shellsuit · Jun 20, 2002, 07:20 AM

Jeez I hope this never happens... OS X is a *nix for christs sake - you want to introduce more cross-platform compatability probs with other *nixs?
When I ssh into a unix box, I expect everything to be where it should be.. not some half-baked, dumbed down mickey-mouser that panders to the lowest common denominator.

Sorry but this has to be said. Apple has made a great job of producing a 'consumer friendly' unix-based OS - the 'holy-grail', so to speak. Don't dumb things down at the base-level please.. never. Some of us require a robust unix OS that behaves as any unix OS should, and that doesn't have to hide it's core from the clueless, or be ashamed of itself in any way. There are OS X users other than consumers.. stop trying to push Apple in a direction which panders to this sector only; it's patronising to say the least, and could end up destroying the very *nixness that contributes a lot to the greatness of OS X.

Cipher13 · Jun 20, 2002, 09:19 AM

<blockquote>quote:<hr />Originally posted by shellsuit:
Jeez I hope this never happens... OS X is a *nix for christs sake - you want to introduce more cross-platform compatability probs with other *nixs?
When I ssh into a unix box, I expect everything to be where it should be.. not some half-baked, dumbed down mickey-mouser that panders to the lowest common denominator.

Sorry but this has to be said. Apple has made a great job of producing a 'consumer friendly' unix-based OS - the 'holy-grail', so to speak. Don't dumb things down at the base-level please.. never. Some of us require a robust unix OS that behaves as any unix OS should, and that doesn't have to hide it's core from the clueless, or be ashamed of itself in any way. There are OS X users other than consumers.. stop trying to push Apple in a direction which panders to this sector only; it's patronising to say the least, and could end up destroying the very *nixness that contributes a lot to the greatness of OS X.<hr /></blockquote>Well, being the Unix genius you so obviously are, I think you can find the folders you need; as opposed to people who want a simple system, and would be SCARED by seeing folders on their drive like 'System' and 'Library' and whatnot.

'Library' is a dumb name as it is for computer newbies to comprehend... "Library? Cool! I wonder what's in there...".

Heh.

It's better to make things a little more difficult for pro's, than it is to scare newbies away.

And hell, if you don't want your system like that, set it back up the standard way. Easy. It can be a 'pro' install option.

This is still the MacOS, more than it is Unix, remember.

Millennium · Jun 20, 2002, 10:19 AM

<blockquote>quote:<hr />For starters, you missed the /etc folder, and I know you can't simply symlink that one out or the system won't boot.<hr /></blockquote>Millenium... type "ls -al /" in the Terminal once, look at etc, and tell me what you see... it's a symlink. Yes, that's right, the etc folder is not at the root level at all - it's in /private. So not only is it true that you can symlink it out, but Apple does symlink it out. I didn't have to create that symlink because one already existed that pointed to /private/etc, and I moved /private and left a symlink in its place so the existing symlink still worked. If the Finder could automatically create the symlinks when dragging a system, it would make the symlinks for etc (and also for tmp, cores, and var). As these symlinks already existed, it's unnecessary for the purposes of this demonstration.[/quote]
Interesting. You're right; had forgotten about this.

How the heck did Apple manage that?
<blockquote>quote:<hr />Yes, I know how hardlinks work, thanks. I actually made a hardlink from the file in /System because I first tried softlinking the file and it didn't work for mach_kernel, and at this point the file was already copied inside /System. For .hidden, I actually did just hard link it from its original location.<hr /></blockquote>You don't understand; by hardlinking, you moved the file back into the root. It is one file, in two places, and the one in /System has no effect at all.

That's the tricky thing about hardlinks, and the thing which is hardest to understand about them. Which is understandable, since no real-world analog to them is possible. If you hardlink a file in /System to the root of the drive, the file actually is at the root of the drive. It's not like an alias or softlink in that respect, where only a pointer to the file would be at the root. In essence, when you made the symlink, you moved mach_kernel back to the root. It is still in /System, too, but it is in /, and that is why the machine could boot.
<blockquote>quote:<hr />As for working across volumes - it doesn't need to! When you drag /System to another volume under my theoretical system, it would copy the contents of /System, including mach_kernel.<hr /></blockquote>Ah, but it would need to find mach_kernel first, then load it, before it could make the copy (I assume you'd want to do this at boot time, so that an accidental copy wouldn't litter the root of the drive with links and such). You have a chicken-and-egg situation there; you must boot before you can make the copy, but you must make the copy before you can boot.

To be honest, I think they should just make /System totally invisible in the Finder, ad add a System Backup Utility. Authenticate, pick a disc to back the system up to, and you're done. While it's not quite as simple as a drag-and-drop, I'm not entirely certain it should be that simple, and this is quite close.

And as I understand it, the backup utility is coming in 10.2. Now they just need to make /System invisible, to cut down on the /System-vs-/Library confusion. Seeing as /System isn't supposed to be modified anyway, what's the point of its being visible?

Drizzt · Jun 20, 2002, 10:26 AM

<blockquote>quote:<hr />Originally posted by Cipher13:
 <blockquote>quote:<hr />Originally posted by shellsuit:
Jeez I hope this never happens... OS X is a *nix for christs sake - you want to introduce more cross-platform compatability probs with other *nixs?
When I ssh into a unix box, I expect everything to be where it should be.. not some half-baked, dumbed down mickey-mouser that panders to the lowest common denominator.

Sorry but this has to be said. Apple has made a great job of producing a 'consumer friendly' unix-based OS - the 'holy-grail', so to speak. Don't dumb things down at the base-level please.. never. Some of us require a robust unix OS that behaves as any unix OS should, and that doesn't have to hide it's core from the clueless, or be ashamed of itself in any way. There are OS X users other than consumers.. stop trying to push Apple in a direction which panders to this sector only; it's patronising to say the least, and could end up destroying the very *nixness that contributes a lot to the greatness of OS X.<hr /></blockquote>Well, being the Unix genius you so obviously are, I think you can find the folders you need; as opposed to people who want a simple system, and would be SCARED by seeing folders on their drive like 'System' and 'Library' and whatnot.

'Library' is a dumb name as it is for computer newbies to comprehend... "Library? Cool! I wonder what's in there...".

Heh.

It's better to make things a little more difficult for pro's, than it is to scare newbies away.

And hell, if you don't want your system like that, set it back up the standard way. Easy. It can be a 'pro' install option.

This is still the MacOS, more than it is Unix, remember.<hr /></blockquote>Just to remind you that these folders will stay there using this hack, what will change is the /usr /etc /private ... folders that are already invisible

Drizzt · Jun 20, 2002, 10:32 AM

<blockquote>quote:<hr />Originally posted by CharlesS:
 <blockquote>quote:<hr />Originally posted by Drizzt:
The reality is different.. I don't want to burst you ballon.. but you did almost nothing..<hr /></blockquote>Yeah, specifically what I did was to test a simple hack that several have mentioned earlier but no one has actually tried. I verified that it works. I'm not claiming to have rewritten the kernel or anything... just proven that Apple could do this without too much trouble if they wanted to. The fact that this was a simple operation was actually part of the point.

<blockquote>quote:<hr />For MacOS 9 speakers.. it would be like this :

You move your Documents folder in the 'System Folder' and put an alias to Documents on the root of your hard drive.

The system acts like before, but your files are in 'System Folder'.<hr /></blockquote>No, in Mac OS 9 terms, it would be more like the System files as well as various extensions and support files were littered at the root level of the drive and made invisible, and I moved them into the System Folder.

<blockquote>quote:<hr />The Unix core understands links correctly, that's why everything works fine.. but the Cocoa API still struggles it seams..<hr /></blockquote>In what way?<hr /></blockquote>I think you didn't understand what I meant. Right now everything's working fine with a bit of luck, because it looks like Cocoa has problem with links (you said it youself with installer.app). The secure and thrusty way to do that is to change internal variables of the system to get this kind of stuff from /System/etc instead of /etc for exemple. This way you won't get invisible links on the root of the system, you won't need them!

Also, these links shouldn't be made by the system, or else a lot of system services won't work correctly and won't boot at all (Apache, SSH).

absmiths · Jun 20, 2002, 10:46 AM

<blockquote>quote:<hr />Originally posted by Millennium:
Ah, but it would need to find mach_kernel first, then load it, before it could make the copy (I assume you'd want to do this at boot time, so that an accidental copy wouldn't litter the root of the drive with links and such). You have a chicken-and-egg situation there; you must boot before you can make the copy, but you must make the copy before you can boot.
<hr /></blockquote>This is not intended to be done at boot-time. The point is for a user who is logged in (hence working on a running system) to drag the System folder to another volume (thereby copying /System/mach_kernel - hence the need for the hardlink), and the Finder would create symbolic links on that volume as needed (creating the mach_kernel hardlink at the volume root), then bless the volume so that it may be seleted as the startup volume.

absmiths · Jun 20, 2002, 10:50 AM

<blockquote>quote:<hr />Originally posted by Drizzt:
I think you didn't understand what I meant. Right now everything's working fine with a bit of luck, because it looks like Cocoa has problem with links (you said it youself with installer.app). The secure and thrusty way to do that is to change internal variables of the system to get this kind of stuff from /System/etc instead of /etc for exemple. This way you won't get invisible links on the root of the system, you won't need them!

Also, these links shouldn't be made by the system, or else a lot of system services won't work correctly and won't boot at all (Apache, SSH).<hr /></blockquote>Installer.app being a cocoa app doesn't have anything to do with Cocoa API's not working with symlinks. The problem was with pax - which is not written in cocoa - and besides it was an explicit decision made by the writers of that package (albeit a dumb one - I can't conceive of any situation where deleting a symbolic link and replacing it with an empty folder would be desirable). So this is not an issue of bad API's or frameworks.

Drizzt · Jun 20, 2002, 10:56 AM

<blockquote>quote:<hr />Originally posted by absmiths:
 <blockquote>quote:<hr />Originally posted by Drizzt:
I think you didn't understand what I meant. Right now everything's working fine with a bit of luck, because it looks like Cocoa has problem with links (you said it youself with installer.app). The secure and thrusty way to do that is to change internal variables of the system to get this kind of stuff from /System/etc instead of /etc for exemple. This way you won't get invisible links on the root of the system, you won't need them!

Also, these links shouldn't be made by the system, or else a lot of system services won't work correctly and won't boot at all (Apache, SSH).<hr /></blockquote>Installer.app being a cocoa app doesn't have anything to do with Cocoa API's not working with symlinks. The problem was with pax - which is not written in cocoa - and besides it was an explicit decision made by the writers of that package (albeit a dumb one - I can't conceive of any situation where deleting a symbolic link and replacing it with an empty folder would be desirable). So this is not an issue of bad API's or frameworks.<hr /></blockquote>Right... forgot about pax..

I see where the problem is..

Let's say there's a link in /etc/smb to /private/system/smb-conf/

Now.. let's say MacOS X uses /private/system/smb-conf/ and unix services uses /etc/smb.

If installer.app updates the SMB package, files needs to be availlable in both directories, but if in the update process it deletes the folder and recreates it.. you're hoosed <img border="0" title="" alt="[Wink]" src="wink.gif" />

absmiths · Jun 20, 2002, 10:58 AM

<blockquote>quote:<hr />Originally posted by shellsuit:
Jeez I hope this never happens... OS X is a *nix for christs sake - you want to introduce more cross-platform compatability probs with other *nixs?
When I ssh into a unix box, I expect everything to be where it should be.. not some half-baked, dumbed down mickey-mouser that panders to the lowest common denominator.
<hr /></blockquote>The compatibility is there - witness the symbolic links. If you think this is unprecedented, take a look at Linux, VMS, HPUX, AIX, IRIX, SCO, SBDI and others and then tell me that they all store stuff in the same places. You can generally hope to find at least nice links to things that are in really weird places, but even very well known things (to Linux users) like /usr/local are sometimes non-existent.

In my experience logging into a new Unix for the first time usually requires a little exploring - you can't just assume that directories will exist where you want them and that they will mean what you expect them to.

Besides, one thing that Unix stinks at is ease of configuration - and if Apple can solve that by creating symlinks or some similar mechanism than I will be all for it.

CharlesS · Jun 20, 2002, 02:27 PM

<blockquote>quote:<hr />Originally posted by shellsuit:
Jeez I hope this never happens... OS X is a *nix for christs sake - you want to introduce more cross-platform compatability probs with other *nixs?
When I ssh into a unix box, I expect everything to be where it should be.. not some half-baked, dumbed down mickey-mouser that panders to the lowest common denominator.

Sorry but this has to be said. Apple has made a great job of producing a 'consumer friendly' unix-based OS - the 'holy-grail', so to speak. Don't dumb things down at the base-level please.. never. Some of us require a robust unix OS that behaves as any unix OS should, and that doesn't have to hide it's core from the clueless, or be ashamed of itself in any way. There are OS X users other than consumers.. stop trying to push Apple in a direction which panders to this sector only; it's patronising to say the least, and could end up destroying the very *nixness that contributes a lot to the greatness of OS X.<hr /></blockquote>This would keep compatibility just as good as it is now. UNIX apps are generally capable of dealing with symlinks. Apple has already partially done this - they moved etc, var, and tmp into /private, and left symlinks at the root. Does this cause problems with any UNIX apps that you know of?

CharlesS · Jun 20, 2002, 02:31 PM

<blockquote>quote:<hr />Originally posted by Drizzt:
I think you didn't understand what I meant. Right now everything's working fine with a bit of luck, because it looks like Cocoa has problem with links (you said it youself with installer.app). The secure and thrusty way to do that is to change internal variables of the system to get this kind of stuff from /System/etc instead of /etc for exemple. This way you won't get invisible links on the root of the system, you won't need them!

Also, these links shouldn't be made by the system, or else a lot of system services won't work correctly and won't boot at all (Apache, SSH).<hr /></blockquote>As someone who has written in Cocoa quite a bit recently, I can tell you first-hand that Cocoa has absolutely no problems with symbolic links (aliases, now are a story for another day). In the case of Installer.app, the blame mostly rests on the pax utility, but there are ways to work around pax's limitations that the authors of Installer.app could have taken advantage of. In my app, for example, I use Cocoa to figure out where files will go instead of letting pax do it, with the result that symlinks are handled much more properly. I know of no other programs that have problems with symlinks besides pax.

How many UNIX services have been crippled by the fact that /etc is a symlink to /private/etc?

CharlesS · Jun 20, 2002, 02:47 PM

<blockquote>quote:<hr />Originally posted by Millennium:
 <blockquote>quote:<hr />Yes, I know how hardlinks work, thanks. I actually made a hardlink from the file in /System because I first tried softlinking the file and it didn't work for mach_kernel, and at this point the file was already copied inside /System. For .hidden, I actually did just hard link it from its original location.<hr /></blockquote>You don't understand; by hardlinking, you moved the file back into the root. It is one file, in two places, and the one in /System has no effect at all.

That's the tricky thing about hardlinks, and the thing which is hardest to understand about them. Which is understandable, since no real-world analog to them is possible. If you hardlink a file in /System to the root of the drive, the file actually is at the root of the drive. It's not like an alias or softlink in that respect, where only a pointer to the file would be at the root. In essence, when you made the symlink, you moved mach_kernel back to the root. It is still in /System, too, but it is in /, and that is why the machine could boot.<hr /></blockquote>Please stop explaining hard links; I already understand how they work, okay? I'm afraid it is you that doesn't understand what I'm doing with them. I will try to explain more clearly:

Simply putting mach_kernel in /System and leaving a soft link behind doesn't work. Apparently, the mach_kernel file needs to be at the root of the drive. This is the entire reason why I use a hard link. The file can be at the root of the drive, so it will work, but it can also be inside /System, so that when the user copies the System directory by dragging it, mach_kernel gets copied as well. Everything can be neatly contained inside /System this way. Otherwise, we'd have to make an exception for mach_kernel. The Finder can create the soft and hard links at the time of the copy.

<blockquote>quote:<hr /> <blockquote>quote:<hr />As for working across volumes - it doesn't need to! When you drag /System to another volume under my theoretical system, it would copy the contents of /System, including mach_kernel.<hr /></blockquote>Ah, but it would need to find mach_kernel first, then load it, before it could make the copy (I assume you'd want to do this at boot time, so that an accidental copy wouldn't litter the root of the drive with links and such). You have a chicken-and-egg situation there; you must boot before you can make the copy, but you must make the copy before you can boot.<hr /></blockquote>See above. The hard links solve this problem.

The links can be made on copying the folder - if a user is in the habit of typing an admin password for an operation that was an accident, a few links on their drive are going to be the least of that user's worries...

<blockquote>quote:<hr />To be honest, I think they should just make /System totally invisible in the Finder, ad add a System Backup Utility. Authenticate, pick a disc to back the system up to, and you're done. While it's not quite as simple as a drag-and-drop, I'm not entirely certain it should be that simple, and this is quite close.

And as I understand it, the backup utility is coming in 10.2. Now they just need to make /System invisible, to cut down on the /System-vs-/Library confusion. Seeing as /System isn't supposed to be modified anyway, what's the point of its being visible?<hr /></blockquote>We've had this discussion before:

1. All the developer documentation and headers are in /System. Not all of it is linked to via /Developer.

2. The folder is write-protected, so users can't modify it; thus, it is harmless.

3. Having an invisible folder that takes up multiple gigabytes of space leaves a user wondering where all their space went.

4. Hiding huge folders with thousands of files is not the Mac Way.

5. The /System directory is not comparable to the System suitcase - rather, it is comparable to parts of the System suitcase, the Finder, various system enablers, and numerous fonts, extensions, control panels, etc. that are preinstalled by the system. There's really no clean analogy to it in OS 9.

6. Even assuming the System suitcase parallel, the System suitcase was not hidden in OS 9.

7. Some users like to understand their system to the best of their abilities. Hiding things from them makes this more difficult.

8. Knowing some things about /System can be very useful - for example, knowing about certain files (such as Extensions.mkext) that are not immutable, but which actually are deleted and recreated at startup, but which can prevent the system from starting up at all if they are corrupted. Also, witness tech support cases such as people whose Classic environment refuses to launch, because the setuid bit was cleared somehow. If the system worked the way you want, the number of people who would be able to help them by telling them the path to Classic Startup, what its permissions should be, and how to set them would be very low...

9. If the System directory were hidden, my idea for being able to transplant it via drag and drop would be impossible.

However, I'm not going to rehash the argument we had earlier. Those are my points, and they're the first and last thing I'm going to say about the subject in this thread. Let's go back to the original topic...

[ 06-20-2002, 03:35 PM: Message edited by: CharlesS ]

Drizzt · Jun 20, 2002, 03:58 PM

<blockquote>quote:<hr />Originally posted by CharlesS:
 <blockquote>quote:<hr />Originally posted by Drizzt:
I think you didn't understand what I meant. Right now everything's working fine with a bit of luck, because it looks like Cocoa has problem with links (you said it youself with installer.app). The secure and thrusty way to do that is to change internal variables of the system to get this kind of stuff from /System/etc instead of /etc for exemple. This way you won't get invisible links on the root of the system, you won't need them!

Also, these links shouldn't be made by the system, or else a lot of system services won't work correctly and won't boot at all (Apache, SSH).<hr /></blockquote>As someone who has written in Cocoa quite a bit recently, I can tell you first-hand that Cocoa has absolutely no problems with symbolic links (aliases, now are a story for another day). In the case of Installer.app, the blame mostly rests on the pax utility, but there are ways to work around pax's limitations that the authors of Installer.app could have taken advantage of. In my app, for example, I use Cocoa to figure out where files will go instead of letting pax do it, with the result that symlinks are handled much more properly. I know of no other programs that have problems with symlinks besides pax.

How many UNIX services have been crippled by the fact that /etc is a symlink to /private/etc?<hr /></blockquote>This was said in the case there is no links...

CharlesS · Jun 20, 2002, 05:00 PM

<blockquote>quote:<hr />Originally posted by Drizzt:
 <blockquote>quote:<hr />Originally posted by CharlesS:
 <blockquote>quote:<hr />Originally posted by Drizzt:
I think you didn't understand what I meant. Right now everything's working fine with a bit of luck, because it looks like Cocoa has problem with links (you said it youself with installer.app). The secure and thrusty way to do that is to change internal variables of the system to get this kind of stuff from /System/etc instead of /etc for exemple. This way you won't get invisible links on the root of the system, you won't need them!

Also, these links shouldn't be made by the system, or else a lot of system services won't work correctly and won't boot at all (Apache, SSH).<hr /></blockquote>As someone who has written in Cocoa quite a bit recently, I can tell you first-hand that Cocoa has absolutely no problems with symbolic links (aliases, now are a story for another day). In the case of Installer.app, the blame mostly rests on the pax utility, but there are ways to work around pax's limitations that the authors of Installer.app could have taken advantage of. In my app, for example, I use Cocoa to figure out where files will go instead of letting pax do it, with the result that symlinks are handled much more properly. I know of no other programs that have problems with symlinks besides pax.

How many UNIX services have been crippled by the fact that /etc is a symlink to /private/etc?<hr /></blockquote>This was said in the case there is no links...<hr /></blockquote>There are links - /etc is a symlink, even out of the box from Apple.

The links are necessary, and environment variables will not cut it. UNIX apps (and UNIX users) expect bin, usr, etc. to be at the root of the drive - if something's not there, you'll break a ton of UNIX apps and shell scripts, not to mention Cocoa and Carbon apps that launch UNIX tools to do various tasks.

Big Mac · Jun 20, 2002, 05:41 PM

Charles you are in the right here. Drag and drop system folder copying is the way the Mac is supposed to work; it's always how the Mac has worked. The only thing I wondered about was the implication concerning traditional Unix application support, but the case you've made (that it has no impact) seems pretty convincing to me.

Keep working on this, and get Apple involved to. Apple should know the Mac experience is very important to its loyal (and vocal) constituency.

CharlesS · Jun 20, 2002, 05:46 PM

<blockquote>quote:<hr />Originally posted by Big Mac:
Charles you are in the right here. Drag and drop system folder copying is the way the Mac is supposed to work; it's always how the Mac has worked. The only thing I wondered about was the implication concerning traditional Unix application support, but the case you've made (that it has no impact) seems pretty convincing to me.

Keep working on this, and get Apple involved to. Apple should know the Mac experience is very important to its loyal (and vocal) constituency.<hr /></blockquote>Unfortunately, I don't think there's much more work I can do on this. The job can only be finished by Apple, unless the Finder becomes open-source (yeah, right!).

I've been planning my feedback letter to Apple based on how my post has been received in this forum - sometime during the next few days I will send it in. I encourage anyone else who is interested in this to do the same.

Charles

cwasko · Jun 21, 2002, 01:35 AM

It seems to me that all the things that ae being discussed here are more Darwin related. Granted, there are Mac items which will be affect by all this, but if the idea is blessed by Apple, then thats a wrap. Dumping all the UNIXy stuff in /System and /Library is a great value to the maintanability of the OS. Brining this up in the Darwin lists or even to the man himself (<a href="http://kerneltrap.org/node.php?id=278" target="_blank">http://kerneltrap.org/node.php?id=278</a>) would be very valuable, in addition to sending feedback.

[ 06-21-2002, 01:36 AM: Message edited by: cwasko ]

absmiths · Jun 21, 2002, 10:53 AM

There is a further consideration - when a user drags the /System folder to a new partition, only the system gets copied, right? No user documents, applications, no home directory, etc. In this case I think the backup utility is a better idea because then the user can copy the system and all of their personal applications, documents, etc, and not have to worry about where Apple has hidden everything.

CharlesS · Jun 21, 2002, 03:14 PM

<blockquote>quote:<hr />Originally posted by absmiths:
There is a further consideration - when a user drags the /System folder to a new partition, only the system gets copied, right? No user documents, applications, no home directory, etc. In this case I think the backup utility is a better idea because then the user can copy the system and all of their personal applications, documents, etc, and not have to worry about where Apple has hidden everything.<hr /></blockquote>Not that hard to pick up Library, Applications, and Users as well. Even in OS 9, you'd need to grab a few extra folders to do a complete backup of your system plus all your apps and documents...

mrl14 · Jun 21, 2002, 08:33 PM

If you do this you run into a permissions problems.

1) you can't copy or delete files owned by root
2) if you copy a folder over to another drive, it gains the ownership of current user logged in. So to do a full backup you'd login as root, copy everything over, when you relogin as your user everything will be owned by root. That's why you need a program like Carbon Copy Cloner which uses "ditto" to remove stuff.

and I know th is because I ran into this exact problem. It took me a while to figure out how i can get files that I OWN to actually be owned by my user.

Bottom line is this idea won't work...its one of the joys of UNIX =) You can't move things if you're using OS X in a multi user environment.

Try this, install OS X from scratch on a new drive. Make a NEW user on that system that is different from the last install (user michael instead of mike). Now boot up X under the old system and login as mike (who currently owns all files in his dir). Drag those files onto the new install...documents, apps, etc. Now restart onto the new install and get info on those files...they are all owned by mike or some main category because mike doesn't exist. Now i know if both users are admins the effects aren't as drastic as if one was an admin user and one is just a regular user.

[ 06-21-2002, 08:37 PM: Message edited by: mrl14 ]

CharlesS · Jun 21, 2002, 11:36 PM

<blockquote>quote:<hr />Originally posted by mrl14:
If you do this you run into a permissions problems.

1) you can't copy or delete files owned by root
2) if you copy a folder over to another drive, it gains the ownership of current user logged in. So to do a full backup you'd login as root, copy everything over, when you relogin as your user everything will be owned by root. That's why you need a program like Carbon Copy Cloner which uses "ditto" to remove stuff.

and I know th is because I ran into this exact problem. It took me a while to figure out how i can get files that I OWN to actually be owned by my user.

Bottom line is this idea won't work...its one of the joys of UNIX =) You can't move things if you're using OS X in a multi user environment.

Try this, install OS X from scratch on a new drive. Make a NEW user on that system that is different from the last install (user michael instead of mike). Now boot up X under the old system and login as mike (who currently owns all files in his dir). Drag those files onto the new install...documents, apps, etc. Now restart onto the new install and get info on those files...they are all owned by mike or some main category because mike doesn't exist. Now i know if both users are admins the effects aren't as drastic as if one was an admin user and one is just a regular user.<hr /></blockquote>We've already covered this - as I said before, I'm thinking of a hypothetical future version of the Finder that allows admin users to authenticate and move/copy things as root. This is far from impossible and is a feature the Finder aught to have anyway...