Welcome to the MacNN Forums.

If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

You are here: MacNN Forums > Software - Troubleshooting and Discussion > macOS > Leopard Technical Stuff thread

Leopard Technical Stuff thread
Thread Tools
CharlesS
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Oct 26, 2007, 10:28 PM
 
Besson3c kept wanting to talk about the technical aspects of Leopard while it was in beta - well, now it's out, so we can talk about this stuff, and here's the thread to do it.

I don't remember most of the questions, but I do remember that at one point he was asking whether HFS+ supported hard links to folders, and whether Leopard exploited this for Time Machine. And the answer is...

... no, it doesn't. What actually happens is that Leopard uses a folder at the root of the drive named .HFS+ Private Data. Each folder that is part of a Time Machine backup is actually stored in this folder, with a generic name. The folders you see in Time Machine are actually aliases - yep, not symlinks, but full-blown aliases, complete with a resource fork and an 'alis' in it - to the folder in .HFS+ Private Data. The OS hides this fact and presents the aliases as actual folders, presumably at a very low level since they actually look like folders even in the Terminal, so what you end up with is something that in practical terms works just like a hard link to a folder, but isn't actually.

All right, next question!

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
arcticmac
Dedicated MacNNer
Join Date: Apr 2004
Status: Offline
Reply With Quote
Oct 26, 2007, 10:47 PM
 
why did they change the installer files to be xar packages?
     
CharlesS  (op)
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Oct 26, 2007, 10:50 PM
 
Well, I don't work for Apple, so I couldn't tell you why they did it. My guess would be that they wanted to make it easier to put a .pkg up for download without having to wrap it in a .dmg or a .zip.

In Leopard, it's actually possible to have a .mpkg point to a bunch of URLs to .pkgs that are on the Web somewhere. Running the installer would download those packages and then install them. Kind of like the way the old QuickTime installer used to work in OS 9.

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
Geobunny
Mac Elite
Join Date: Oct 2000
Location: Edinburgh, Scotland
Status: Offline
Reply With Quote
Oct 27, 2007, 08:19 AM
 
Originally Posted by CharlesS View Post
Well, I don't work for Apple, so I couldn't tell you why they did it. My guess would be that they wanted to make it easier to put a .pkg up for download without having to wrap it in a .dmg or a .zip.
Good point, I never thought of it like that. I just assumed it was to stop people showing package contents and playing around with the requirements dict to make stuff install where it's not supposed to!

In Leopard, it's actually possible to have a .mpkg point to a bunch of URLs to .pkgs that are on the Web somewhere. Running the installer would download those packages and then install them. Kind of like the way the old QuickTime installer used to work in OS 9.
Hey that's really cool, I must've missed that one. Mind you, there are 4 ADC videos re the installer which I've not watched yet so I can't really blame anyone but myself
ClamXav - the free virus scanner for Mac OS X | Geobunny learns to fly
     
ginoledesma
Mac Elite
Join Date: Apr 2000
Location: Los Angeles, CA
Status: Offline
Reply With Quote
Oct 27, 2007, 10:15 AM
 
CharlesS, does that imply that Time Machine works on HFS+ volumes only then? If so, this will explain Apple's very explicit working with remote drive support for TM (i.e. other Apple Macs sharing the drive). I was hoping that I could use my existing NAS to expose its share using AFP instead of CIFS. But as the underlying filesystem is ext3, that could cause problems down the line (e.g. allowed characters in filenames).
     
CharlesS  (op)
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Oct 27, 2007, 11:02 AM
 
Originally Posted by ginoledesma View Post
CharlesS, does that imply that Time Machine works on HFS+ volumes only then?
I dunno, because I haven't tried it on other FS types. It would certainly be possible for Apple to do the same trick with symbolic links on another file system that they do with aliases on HFS+. I'd be kind of surprised if they did, though.

btw: why hasn't besson3c shown up in this thread yet?

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
Geobunny
Mac Elite
Join Date: Oct 2000
Location: Edinburgh, Scotland
Status: Offline
Reply With Quote
Oct 27, 2007, 11:59 AM
 
Originally Posted by CharlesS View Post
I dunno, because I haven't tried it on other FS types. It would certainly be possible for Apple to do the same trick with symbolic links on another file system that they do with aliases on HFS+. I'd be kind of surprised if they did, though.
It would've thought that it should work actually. AFP is just a protocol and AFAIR the underlying file system is more or less immaterial. Where I used to work, we had slackware linux servers using ext3 as the file system but directories were shared out by NetATalk (opensource AFP server) to Macs which, to all intents and purposes thought they were working with regular AFP shares. The point is, resource forks were maintained through a .AppleDouble directory and aliases worked flawlessly. [until the .AppleDouble directory got corrupt but I'm still convinced that's just cos we tinkered with the source code before deploying NetATalk!]
ClamXav - the free virus scanner for Mac OS X | Geobunny learns to fly
     
CharlesS  (op)
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Oct 27, 2007, 12:02 PM
 
Well, maybe it will. Only way to find out would be to try it.

The one thing is that while resource forks are a thing that are pretty easy to implement on other file systems through the use of dotfiles, aliases aren't, so if there's some sort of backup implementation (like using symbolic links) it could work, but AFAIK it couldn't be implemented the exact same way.

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
Geobunny
Mac Elite
Join Date: Oct 2000
Location: Edinburgh, Scotland
Status: Offline
Reply With Quote
Oct 27, 2007, 12:32 PM
 
Originally Posted by CharlesS View Post
The one thing is that while resource forks are a thing that are pretty easy to implement on other file systems through the use of dotfiles, aliases aren't, so if there's some sort of backup implementation (like using symbolic links) it could work, but AFAIK it couldn't be implemented the exact same way.
Isn't that the whole purpose of defining a protocol though? It allows someone to do a completely different implementation and still be compatible? Well, in theory anyway!

You're right, the best thing is to try it and see what happens.
ClamXav - the free virus scanner for Mac OS X | Geobunny learns to fly
     
Big Mac
Clinically Insane
Join Date: Oct 2000
Location: Los Angeles
Status: Offline
Reply With Quote
Oct 30, 2007, 01:40 PM
 
I missed this thread, too. Anyone want to discuss the technical aspects of the security improvements that are said to block old code injection methods? I know Unsanity apps check for 10.5 and disable themselves, but presumably they do so because the underlying method is no longer supported at all.

"The natural progress of things is for liberty to yield and government to gain ground." TJ
     
CharlesS  (op)
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Oct 30, 2007, 01:58 PM
 
Well, I'm a little hesitant to talk about that, lest the people from Unsanity read this thread and use it for clues on finding ways to work around stuff.

One thing I will say, though, is that Leopard has signed code. And you can check the integrity of the PID of a running process, rather than just being able to check the binary on disk. If this works the way I hope it does, it should be able to alert you if something like APE tries to rewrite your code in memory. We'll find out, I guess, as soon as Unsanity finds a new attack vector or API to abuse in order to make patching work again. And hopefully once that happens, Apple will close whatever hole they end up exploiting, because patching is such a massive security hole, especially in Leopard.

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
Big Mac
Clinically Insane
Join Date: Oct 2000
Location: Los Angeles
Status: Offline
Reply With Quote
Oct 30, 2007, 02:00 PM
 
Especially in Leopard? As in being a bigger hole than in Tiger, which was more or less wide open?

"The natural progress of things is for liberty to yield and government to gain ground." TJ
     
CharlesS  (op)
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Oct 30, 2007, 02:08 PM
 
Well, for one thing, the firewall in Leopard is application-specific instead of blocking incoming requests by port. It now works for incoming connections much the same way that Little Snitch works for outgoing connections - if an app tries to open a port, it pops up a dialog warning you about it, and if you choose to allow it, it adds the app to a whitelist. This means that once you've allowed some application to open ports on your machine, it becomes just as easy to get around the firewall and open whatever ports you want by patching that app as it was to get around Little Snitch by patching some trusted network app (like Safari) in Tiger. This means, of course, that the firewall is completely useless for anything other than providing a false sense of security as long as patching is possible. Which isn't a good thing.

This is not to say that patching wasn't a massive security hole in Tiger - it's just worse in Leopard. And hopefully Apple will recognize that and take steps to prevent it from occurring. We'll see. It certainly would be nice not to receive crash reports in the mail all the time anymore from crashes that were caused by APE.

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Oct 30, 2007, 02:22 PM
 
Originally Posted by CharlesS View Post
Besson3c kept wanting to talk about the technical aspects of Leopard while it was in beta - well, now it's out, so we can talk about this stuff, and here's the thread to do it.

I don't remember most of the questions, but I do remember that at one point he was asking whether HFS+ supported hard links to folders, and whether Leopard exploited this for Time Machine. And the answer is...

... no, it doesn't. What actually happens is that Leopard uses a folder at the root of the drive named .HFS+ Private Data. Each folder that is part of a Time Machine backup is actually stored in this folder, with a generic name. The folders you see in Time Machine are actually aliases - yep, not symlinks, but full-blown aliases, complete with a resource fork and an 'alis' in it - to the folder in .HFS+ Private Data. The OS hides this fact and presents the aliases as actual folders, presumably at a very low level since they actually look like folders even in the Terminal, so what you end up with is something that in practical terms works just like a hard link to a folder, but isn't actually.

All right, next question!


Interesting... So, what happens if this text file is misplaced or corrupted? Is there a way to rebuild this file? Is this file used for searching? If so, wouldn't unindexed data be pretty slow? Or, is this text file used to build a flat file database?

Does Apple ever use flat file DBs? Berklee DB? Skiplist? MySQLite?
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Oct 30, 2007, 02:23 PM
 
Originally Posted by arcticmac View Post
why did they change the installer files to be xar packages?
Xar is actually a very cool format... It supports a number of features, including security/encryption features.
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Oct 30, 2007, 02:24 PM
 
Originally Posted by ginoledesma View Post
CharlesS, does that imply that Time Machine works on HFS+ volumes only then? If so, this will explain Apple's very explicit working with remote drive support for TM (i.e. other Apple Macs sharing the drive). I was hoping that I could use my existing NAS to expose its share using AFP instead of CIFS. But as the underlying filesystem is ext3, that could cause problems down the line (e.g. allowed characters in filenames).
I hope that somebody figures out a way to use TM with sshfs mounted drives. I'm a fan of sshfs
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Oct 30, 2007, 02:26 PM
 
Originally Posted by CharlesS View Post
btw: why hasn't besson3c shown up in this thread yet?


Simply didn't notice the thread...

Mods: this is a good thread, please don't lock it cause my name is in the title, I don't mind at all, FWIW...
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Oct 30, 2007, 02:27 PM
 
Originally Posted by CharlesS View Post
Well, maybe it will. Only way to find out would be to try it.

The one thing is that while resource forks are a thing that are pretty easy to implement on other file systems through the use of dotfiles, aliases aren't, so if there's some sort of backup implementation (like using symbolic links) it could work, but AFAIK it couldn't be implemented the exact same way.

Here's a question for you Charles:

Any decisions to move away from metadata littering and towards more of a file system approach using something like xattr?
     
Big Mac
Clinically Insane
Join Date: Oct 2000
Location: Los Angeles
Status: Offline
Reply With Quote
Oct 30, 2007, 02:29 PM
 
Siracusa covers that topic, besson.

"The natural progress of things is for liberty to yield and government to gain ground." TJ
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Oct 30, 2007, 02:32 PM
 
Which topic?

(Yes, I haven't thoroughly read the more technical parts of Siracusa's article yet... Haven't had time to give the whole thing the attention it deserves)
     
Art Vandelay
Professional Poster
Join Date: Sep 2002
Location: New York, NY
Status: Offline
Reply With Quote
Oct 30, 2007, 02:37 PM
 
The one you just asked... using xattr.
Vandelay Industries
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Oct 30, 2007, 02:43 PM
 
Ahhh... thanks, never mind then, I'll check back in once I've read this part of the article.
     
Peter
Addicted to MacNN
Join Date: Oct 2002
Location: England | San Francisco
Status: Offline
Reply With Quote
Oct 30, 2007, 05:57 PM
 
gonna remove the "besson3c" from the title, sorrrry...
we don't have time to stop for gas
     
CharlesS  (op)
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Oct 30, 2007, 07:50 PM
 
Originally Posted by besson3c View Post
Interesting... So, what happens if this text file is misplaced or corrupted? Is there a way to rebuild this file? Is this file used for searching? If so, wouldn't unindexed data be pretty slow? Or, is this text file used to build a flat file database?

Does Apple ever use flat file DBs? Berklee DB? Skiplist? MySQLite?
The post you quoted had no mention of text files. If you were referring to the .HFS+ Private Data folder, I have no idea what would happen if you renamed that, and I'm not willing to find out! Since the "hard links" to directories are aliases rather than symbolic links, I'm guessing they'd still point to the correct folder, but as to whether they'd still appear to be hard links, I don't know!

Apple does use SQLite for the Spotlight index though IIRC...

Originally Posted by besson3c View Post
Here's a question for you Charles:

Any decisions to move away from metadata littering and towards more of a file system approach using something like xattr?
I'm not an Apple employee! I don't know what Apple's future plans are. If I did, I'd probably not be at liberty to discuss them anyway.

With that said, it's a pretty good guess that they will, though, as Leopard does make use of extended attributes for some things that used to be in dotfiles (read Siracusa's article for some examples - it's really a nice article with lots of info).

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
   
Thread Tools
 
Forum Links
Forum Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Top
Privacy Policy
All times are GMT -4. The time now is 05:27 PM.
All contents of these forums © 1995-2017 MacNN. All rights reserved.
Branding + Design: www.gesamtbild.com
vBulletin v.3.8.8 © 2000-2017, Jelsoft Enterprises Ltd.,