Welcome to the MacNN Forums.

If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

You are here: MacNN Forums > Community > MacNN Lounge > Torvalds calls OS X's HFS+ "complete and utter crap". - was "I swear he's an idiot."

Torvalds calls OS X's HFS+ "complete and utter crap". - was "I swear he's an idiot." (Page 2)
Thread Tools
ghporter
Administrator
Join Date: Apr 2001
Location: San Antonio TX USA
Status: Offline
Reply With Quote
Feb 7, 2008, 10:03 AM
 
Originally Posted by Dakar the Fourth View Post
I would have preferred "Sorry Tuxedo Fans"
Too soft and not very clear on what iomatic was commenting on. I kept his original title at the end so his first post makes sense (ish).

Glenn -----OTR/L, MOT, Tx
     
Dakar the Fourth
Addicted to MacNN
Join Date: Nov 2007
Location: In the hearts and minds of MacNNers
Status: Offline
Reply With Quote
Feb 7, 2008, 10:06 AM
 
I wasn't being serious, gh.

Well, I guess I would seriously prefer it, but I wouldn't seriously expect someone to consider it.
Seriously.
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Feb 7, 2008, 10:51 AM
 
How about "I can't believe I had sex with that f-ing penguin"?
     
sek929
Posting Junkie
Join Date: Nov 1999
Location: Cape Cod, MA
Status: Offline
Reply With Quote
Feb 7, 2008, 10:53 AM
 
Originally Posted by Dakar the Fourth View Post
I wasn't being serious, gh.

Well, I guess I would seriously prefer it, but I wouldn't seriously expect someone to consider it.
Seriously.
Wow, someone is serious.

But seriously...
     
turtle777
Clinically Insane
Join Date: Jun 2001
Location: planning a comeback !
Status: Offline
Reply With Quote
Feb 7, 2008, 11:49 AM
 
Originally Posted by besson3c View Post
How about "I can't believe I had sex with that f-ing penguin"?


-t
     
The Godfather
Addicted to MacNN
Join Date: Dec 1999
Location: Tampa, Florida
Status: Offline
Reply With Quote
Feb 7, 2008, 01:21 PM
 
The only newsworthy thing about this thread is the fanboist response of ignorant fanbois such as the OP, if that.
     
The Godfather
Addicted to MacNN
Join Date: Dec 1999
Location: Tampa, Florida
Status: Offline
Reply With Quote
Feb 8, 2008, 10:01 PM
 
At least it is not FAT32, which is required for OSX-Windows drive swapping, but can't contain any file larger than 4GB.
     
nonhuman
Posting Junkie
Join Date: Jun 2001
Location: Baltimore, MD
Status: Offline
Reply With Quote
Feb 8, 2008, 11:31 PM
 
How is HFS+ OS X's? Didn't it come out with like System 8 (or were we calling it OS 8 by then)? HFSJ and HFSX could be called OS X's, I suppose.
     
CharlesS
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Feb 8, 2008, 11:38 PM
 
Because HFS+ is the file system that OS X uses?

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
nonhuman
Posting Junkie
Join Date: Jun 2001
Location: Baltimore, MD
Status: Offline
Reply With Quote
Feb 8, 2008, 11:51 PM
 
Originally Posted by CharlesS View Post
Because HFS+ is the file system that OS X uses?
But OS X isn't the only OS that uses HFS+ and HFS+ isn't the only file system that OS X uses, nor did HFS+ originate from or because of OS X... Oh well, it's an irrelevant objection anyway...
     
CharlesS
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Feb 9, 2008, 12:20 AM
 
HFS+ is the only file system that you are recommended to use as an OS X boot disk, so I'd say it pretty much is OS X's file system. It's also the only currently shipping OS that uses HFS+ (excluding third party solutions such as MacDrive), although that really doesn't have much to do with whether HFS+ is OS X's file system or not.

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
Chuckit
Clinically Insane
Join Date: Oct 2001
Location: San Diego, CA, USA
Status: Offline
Reply With Quote
Feb 9, 2008, 01:24 AM
 
Originally Posted by nonhuman View Post
How is HFS+ OS X's? Didn't it come out with like System 8 (or were we calling it OS 8 by then)? HFSJ and HFSX could be called OS X's, I suppose.
And Cocoa came out with NeXTstep, but I think most people would agree that Cocoa is a Mac OS X programming framework.

Also, as far as I'm aware, "HFSJ" and "HFSX" are just names for HFS+ with certain features enabled.
Chuck
___
"Instead of either 'multi-talented' or 'multitalented' use 'bisexual'."
     
wataru
Addicted to MacNN
Join Date: Oct 2001
Location: Yokohama, Japan
Status: Offline
Reply With Quote
Feb 9, 2008, 07:51 AM
 
Originally Posted by OreoCookie View Post
[Under a hypothetical ZFS implementation] notebook users can use Time Machine to roll back files without being connected to their external backup drive.
How does that work?
     
OreoCookie
Moderator
Join Date: May 2001
Location: Hilbert space
Status: Offline
Reply With Quote
Feb 9, 2008, 08:15 AM
 
They have posted a long technical explanation on the project's website (it's on video even), but I'll make an attempt at explaining it.

The first thing you have to know is that ZFS never overwrites data: if you save an existing file, it doesn't overwrite the old blocks the old version of the file is saved on, but writes them onto another portion of the disk. Once it has completed writing the new version of the file onto the disk, it changes the directory's reference to the new file and finally writes the changes into the so-called ueberblock which contains the whole directory tree information.

In short it writes the new file, specifies its location within the directory tree and then it updates the global directory tree.

To make a snapshot, ZFS simply does two things: it marks all blocks necessary for the snapshot as non-allocable (i. e. they cannot be overwritten by new data) and it copies the old ueberblock. With these two things, you can easily go back and forth in time. It's important to note that this happens on the block-level, so if you change one bit of your 1 GB video file, only the block containing this one single bit needs to be saved for a new snapshot. Right now, Time Machine has to copy the whole file. (This is an issue for people who use Entourage: all mails are written into one big file.)

This is something you cannot easily do with HFS+ (if at all, although I lack the expertise to rule that out). With ZFS, it's quite easy and built right into the file system tools. This would be great for road warriors like me: I could roll back in time without having my external harddrive at hand!

As you can see, ZFS has real-life advantages if Apple chooses to implement those features (which I think it will, given the high probability it will switch to ZFS and features we have now -- Time Machine).
I don't suffer from insanity, I enjoy every minute of it.
     
mduell
Posting Junkie
Join Date: Oct 2005
Location: Houston, TX
Status: Offline
Reply With Quote
Feb 9, 2008, 10:11 AM
 
Free/open/libre software guys aren't generally known for their tact or modesty; not excusing his behavior, just commenting on it. But at the same time let's not pretend HFS+ is up there with ZFS or some of the other modern filesystems; it's more like exFAT+journaling or ext3.

Originally Posted by CharlesS View Post
I think the thing that takes so long is making all those hard links, and then deleting all those hard links during the pruning process. I've noticed that the pre and post-backup thinning tends to take up the lion's share of the backup process on my machine. And when it stalls for 5 minutes in the middle of backing up 800 KB? It's gotta be that it's busy making all the hard links for the files that aren't getting backed up, and I'm not sure that it's fixable as long as we're using HFS+.
They're not hard links, they're kernel-level symbolic links, because, of course, HFS+ doesn't support hard links.
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Feb 9, 2008, 12:11 PM
 
I'm pretty sure that HFS+ is not as nice as ext3 either...
     
ryaxnb
Grizzled Veteran
Join Date: Sep 2003
Location: Felton, CA
Status: Offline
Reply With Quote
Feb 9, 2008, 01:27 PM
 
To be fair, what other filesystems were created in 1998? Linux users were running ext2, and it lacks many features of HFS+ such as forks, file/volume sizes in the EiBs, Unicode, support of any character in filename, including / and NUL, native Creation timestamp, and B* Trees. Windows users had FAT32 (ick, ick, ick) on most non-NT systems, or even FAT16. NT 4 (which only businesses used) had a primitive version NTFS, nowhere near as good as the NTFS that shipped with 2000, although probably better than HFS+. HFS+ was an easy modification of HFS, like FAT32 and VFAT to the original DOSFAT, but was infinitely more complex than any of those and even better than ext2. It's just that everyone else upgraded their systems (ReiserFS, Reiser4 on Linux, NTFS2000 on Windows 2000), while we stuck with HFS+ during the Mac OS X transition because it was way better than what other users were using in 1998.
Trainiable is to cat as ability to live without food is to human.
Steveis... said: "What would scammers do with this info..." talking about a debit card number!
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Feb 9, 2008, 02:35 PM
 
It's not only the 4 file systems that ryaxnb has listed that have surpassed HFS+, there are many more. In fact, really, you could probably make the argument that HFS+ is at the very bottom of the list of modern file systems in use today.
     
starman
Clinically Insane
Join Date: Jun 2000
Location: Union County, NJ
Status: Offline
Reply With Quote
Feb 9, 2008, 02:50 PM
 
Oreo, that sounds rather inefficient. Eventually, you'll had a disk that's fragmented to hell.

Home - Twitter - Sig Wall-Retired - Flickr
     
CharlesS
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Feb 9, 2008, 05:08 PM
 
Originally Posted by mduell View Post
They're not hard links, they're kernel-level symbolic links, because, of course, HFS+ doesn't support hard links.
Huh? Yes it does.

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
mduell
Posting Junkie
Join Date: Oct 2005
Location: Houston, TX
Status: Offline
Reply With Quote
Feb 9, 2008, 08:17 PM
 
Originally Posted by CharlesS View Post
Huh? Yes it does.
The Challenges of Integrating the Unix and Mac OS Environments

HFS+ lacks support in the volume format for hard links, a standard feature of UFS. Initially, the attempted creation of a link to a file would yield a "not supported" error. We had discussed some "80%" solutions, such as creating symbolic links instead, but the semantics of symbolic links are significantly different. For troubleshooting reasons it is preferable to fail at link creation time than at some later time due to problems related to these semantic differences. The problem is that there is a significant amount of software which breaks if hard link creation fails, and some of that software needs to be redesigned if hard links cannot be used. In order to accommodate this software, we now emulate hard links by creating a "kernel-level" symbolic link which is visible only to and interpreted by the HFS+ file system. This was necessary due to the lack of support in the volume format. The resulting behavior is very similar to that of hard links when viewed from above the kernel, though they are relatively inefficient in comparison.

The author "... is presently a senior software engineer at Apple Computer in Cupertino, California. He works primarily on the BSD subsystem in Mac OS X as a member of the Core Operating System group, and as engineering lead for Apple's open source projects."
     
CharlesS
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Feb 9, 2008, 08:31 PM
 
So the hard links I can make in OS X are fake, then? Huh, I knew it was that way for the "hard links" to directories, but I didn't know that was the case for the regular file ones.

Well, I guess that explains why Time Machine is so dog slow at creating the hard links it uses for its incremental backups, then. Argh, bring on ZFS soon, please!

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
Tomchu
Mac Elite
Join Date: Sep 2005
Status: Offline
Reply With Quote
Feb 9, 2008, 10:41 PM
 
Originally Posted by CharlesS View Post
Try making a script to run through one of those Time Machine backups and make a hard link to every file in them, and see how long it takes.
Time Machine is not as granular as you think. FSEvents only keeps track of changes at the directory level so as to minimize the number of hard links that TM needs to create.

Also, that article about hard links and HFS+ was written in 2000. Could Apple have come up with a more efficient solution to this in a more compatible manner?
     
CharlesS
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Feb 9, 2008, 10:54 PM
 
Originally Posted by Tomchu View Post
Time Machine is not as granular as you think. FSEvents only keeps track of changes at the directory level so as to minimize the number of hard links that TM needs to create.
I know - that post was more in reply to besson3c's "hard links are fast" than anything else.

Also, that article about hard links and HFS+ was written in 2000. Could Apple have come up with a more efficient solution to this in a more compatible manner?
I doubt it, since I know for a fact that they're faking the hard links to directories using aliases (a "hard link" to a directory is really an alias pointing to a folder in /.HFS+ Private Data that the OS interprets as a hard link). Most likely, Apple thought they were going to have ZFS support ready for Leopard, but it wasn't ready for 10.5.0. Time Machine seems like it would be a perfect match for ZFS.

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Feb 9, 2008, 11:33 PM
 
Why has been Apple sitting around on their asses and letting their file system situation slip to the point where it has? Why is this not more of a priority with them?
     
Chuckit
Clinically Insane
Join Date: Oct 2001
Location: San Diego, CA, USA
Status: Offline
Reply With Quote
Feb 9, 2008, 11:41 PM
 
Originally Posted by besson3c View Post
Why has been Apple sitting around on their asses and letting their file system situation slip to the point where it has? Why is this not more of a priority with them?
My guess: Because it really doesn't matter that much to most people. HFS+ is sufficient for the vast majority of common use cases. HFS+ is not the fastest sports car out there, but it gets you from point A to point B just fine. Thus, other things that actually impact users' daily lives and things that somebody would buy an OS for got pushed to the top of the pile.
Chuck
___
"Instead of either 'multi-talented' or 'multitalented' use 'bisexual'."
     
CharlesS
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Feb 10, 2008, 12:10 AM
 
Originally Posted by Chuckit View Post
Thus, other things that actually impact users' daily lives and things that somebody would buy an OS for got pushed to the top of the pile.
My guess would be that it was the iPhone got pushed to the top of the pile.

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
mduell
Posting Junkie
Join Date: Oct 2005
Location: Houston, TX
Status: Offline
Reply With Quote
Feb 10, 2008, 12:43 AM
 
Originally Posted by Tomchu View Post
Also, that article about hard links and HFS+ was written in 2000. Could Apple have come up with a more efficient solution to this in a more compatible manner?
I suppose someone could email Wilfredo Sánchez and ask.
My guess is no; if they could retrofit hard links into HFS, they would have done it back when they came up with the documented hack.
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Feb 10, 2008, 01:56 AM
 
Originally Posted by Chuckit View Post
My guess: Because it really doesn't matter that much to most people. HFS+ is sufficient for the vast majority of common use cases. HFS+ is not the fastest sports car out there, but it gets you from point A to point B just fine. Thus, other things that actually impact users' daily lives and things that somebody would buy an OS for got pushed to the top of the pile.

The extra security measures that Apple put in place don't matter much to most people either, nor does support for specifications which aren't widely used yet (such as IPv6). There are some things an OS vendor has a responsibility to develop out of knowing what is best suited for the user, not simply in response to an expressed interest in a precise feature.

There would be many benefits from a better file system that users would benefit from, such as performance.
     
Big Mac
Clinically Insane
Join Date: Oct 2000
Location: Los Angeles
Status: Offline
Reply With Quote
Feb 10, 2008, 02:37 AM
 
It doesn't sound like ZFS would cause pretty serious fragmentation. I wonder how they get around that.

As for the question why not a new filesystem, I think it also has to do with the fact that new filesystems are rather disruptive technologies. They require work on the part of developers and users. I think it's a wonderful thing that Apple didn't rush us into ZFS. If you thought the Tiger -> Leopard transition has been relatively rocky, you can just imagine how difficult people would find it if Apple had decided to throw a new filesystem at us at the same time. I bet ZFS support will be introduced as a point upgrade in Leopard, and then as developers get comfortable with it we'll see a greater push toward widespread adoption with 10.6.

"The natural progress of things is for liberty to yield and government to gain ground." TJ
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Feb 10, 2008, 02:40 AM
 
Big Mac: fair enough, but how long has ZFS been out for?
     
CharlesS
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Feb 10, 2008, 03:50 AM
 
Originally Posted by Big Mac View Post
I bet ZFS support will be introduced as a point upgrade in Leopard
Wouldn't that violate Sarbanes-Oxley though?

Unfortunately, it's very possible that we could be stuck with not getting ZFS until 10.6. And that sucks. Unlike you, I don't really think there was any reason that Apple didn't include ZFS other than that they simply couldn't get it working in time. I also don't think there's any upside to it - including ZFS in 10.5 wouldn't have rushed anyone into anything - no one would have been forcing you to reformat your hard drive as ZFS. You could have just kept it in HFS+, and you'd have exactly the same product you have now. The way it is, we're probably stuck with HFS+ for another two years.

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
Chuckit
Clinically Insane
Join Date: Oct 2001
Location: San Diego, CA, USA
Status: Offline
Reply With Quote
Feb 10, 2008, 04:37 AM
 
Originally Posted by besson3c View Post
The extra security measures that Apple put in place don't matter much to most people either, nor does support for specifications which aren't widely used yet (such as IPv6). There are some things an OS vendor has a responsibility to develop out of knowing what is best suited for the user, not simply in response to an expressed interest in a precise feature.

There would be many benefits from a better file system that users would benefit from, such as performance.
I do not think ZFS offers any particular benefit in the average case. Even in my case, which is more intensive and technical than the average user, I still don't think it would make that much difference to me.

Also, IPv6 was not introduced in Leopard, so I'm not sure how that's relevant. If that were on the Leopard wishlist, I imagine it very well might have gotten left behind too. We all know Apple was behind schedule due to the iPhone, so they presumably had to go into triage mode, which meant they had to let the filesystem deal with its runny nose by itself.
( Last edited by Chuckit; Feb 10, 2008 at 06:49 AM. )
Chuck
___
"Instead of either 'multi-talented' or 'multitalented' use 'bisexual'."
     
Cipher13
Registered User
Join Date: Apr 2000
Status: Offline
Reply With Quote
Feb 10, 2008, 05:10 AM
 
IPv6 has been supported since Panther (or since pre-OS X on FreeBSD via KAME).
     
mduell
Posting Junkie
Join Date: Oct 2005
Location: Houston, TX
Status: Offline
Reply With Quote
Feb 10, 2008, 11:50 AM
 
Originally Posted by CharlesS View Post
Wouldn't that violate Sarbanes-Oxley though?
No; adding features for free is not a SOX violation, you just have to properly account for it.
     
CharlesS
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Feb 10, 2008, 12:16 PM
 
Originally Posted by Chuckit View Post
I do not think ZFS offers any particular benefit in the average case. Even in my case, which is more intensive and technical than the average user, I still don't think it would make that much difference to me.
I'd say any user who has ever suffered data loss or had to run a disk utility on their drive or reformat it because of directory damage would benefit from ZFS, which is much less prone to these problems.

I'd also say that pretty much all users would benefit from faster speeds in the disk I/O, especially where Time Machine is concerned. You don't have to be an advanced user to get frustrated by Time Machine slowing your work down, or by your having to wait 15 minutes to disconnect your laptop from your hard drive to take it somewhere because Time Machine is in the middle of its backup, which takes ridiculous amounts of time to backup a tiny amount of data, and slows your computer to a crawl while you're doing it. I don't want to even think about what it must be like to try to play a 3D game on Leopard with Time Machine turned on.

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
TETENAL
Addicted to MacNN
Join Date: Aug 2004
Location: FFM
Status: Offline
Reply With Quote
Feb 10, 2008, 01:22 PM
 
Time Machine does not slow your work down. It works in the background and consumes no noticeable resources.
Time Machine does not need 15 minutes to back up incrementally for me. Unless after a large software update that created hundreds of MB to back up, it takes something like 2 minutes.
Even when it's in the middle of a backup you can eject the external backup drive and take your notebook elsewhere.

All that is on HFS+. What you describe is simply not generally true.
     
CharlesS
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Feb 10, 2008, 01:32 PM
 
Originally Posted by TETENAL View Post
Time Machine does not slow your work down. It works in the background and consumes no noticeable resources.


What machine do you have, an octo core Mac Pro or something? On my single G5 iMac, it most definitely does slow my work down, and you simply can't claim it consumes no noticeable resources. I can't tell you how many times I've had the following sequence happen: 1) wonder why the heck a compile is going so slowly, or why my video iChat session with someone just got choppy enough that you could measure its performance in seconds per frame rather than frames per second, causing the person on the other end to ask "what happened?", 2) notice that the external drive's light is flashing, 3) manually stop the Time Machine backup, 4) notice that I now have my performance back.

It gets to the point where if I am doing something and I notice the drive spinning up, I groan to myself "Oh, not again..." It really shouldn't be that way.

Time Machine does not need 15 minutes to back up incrementally for me. Unless after a large software update that created hundreds of MB to back up, it takes something like 2 minutes.
How many files do you have on your hard drive? 3? 4? On mine, 15 minutes is an average - some backups manage to take as little as 10, but then sometimes it takes like 30 or 40 minutes to backup something like 3 MB. It's insane. I posted some logs in a previous thread if you want to see some evidence.

edit: here's the thread with the logs in it. It's gotten somewhat better since I deleted a bunch of stuff from my hard drive, so part of the problem definitely has to do with TM not being able to handle large numbers of files (and I'm blaming HFS+ for this, since a huge amount of time is spent just making the hard links for files that didn't change).

Even when it's in the middle of a backup you can eject the external backup drive and take your notebook elsewhere.
That would disrupt the backup, no?
( Last edited by CharlesS; Feb 10, 2008 at 01:46 PM. )

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
TETENAL
Addicted to MacNN
Join Date: Aug 2004
Location: FFM
Status: Offline
Reply With Quote
Feb 10, 2008, 01:47 PM
 
On my Dual-G5 there is no noticeable slow down. It has 763.039 files on the boot volume. Here's the backup log:

10.02.08 18:36:02 /System/Library/CoreServices/backupd[8834] Backup requested by user
10.02.08 18:36:02 /System/Library/CoreServices/backupd[8834] Starting standard backup
10.02.08 18:36:02 /System/Library/CoreServices/backupd[8834] Backing up to: /Volumes/Backup/Backups.backupdb
10.02.08 18:36:04 /System/Library/CoreServices/backupd[8834] No pre-backup thinning needed: 729.8 MB requested (including padding), 222.36 GB available
10.02.08 18:36:13 /System/Library/CoreServices/backupd[8834] Copied 613 files (3.5 MB) from volume Festplatte.
10.02.08 18:36:13 /System/Library/CoreServices/backupd[8834] Starting post-backup thinning
10.02.08 18:36:18 /System/Library/CoreServices/backupd[8834] Deleted backup /Volumes/Backup/Backups.backupdb/G5/2008-02-09-180552: 222.36 GB now available
10.02.08 18:36:18 /System/Library/CoreServices/backupd[8834] Post-back up thinning complete: 1 expired backups removed
10.02.08 18:36:18 /System/Library/CoreServices/backupd[8834] Backup completed successfully.


So there you have it. One increment took 16 seconds. A regular one is in the order of 2 minutes I would say. I would say I'm content with the performance of HFS+ here.

And yes, disconnecting the backup drive would disrupt the backup obviously. But if you got to go you got to go. Just make a backup when you come back again then. One missed hourly backup is not the end of the world.
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Feb 10, 2008, 01:51 PM
 
Originally Posted by Chuckit View Post
I do not think ZFS offers any particular benefit in the average case. Even in my case, which is more intensive and technical than the average user, I still don't think it would make that much difference to me.

Also, IPv6 was not introduced in Leopard, so I'm not sure how that's relevant. If that were on the Leopard wishlist, I imagine it very well might have gotten left behind too. We all know Apple was behind schedule due to the iPhone, so they presumably had to go into triage mode, which meant they had to let the filesystem deal with its runny nose by itself.
The iPhone theory makes sense, but my point about IPv6 was that it is an advanced feature that Apple has chosen to spend time on that does not benefit the average user. There are dozens of other examples.

See my post I'm about to write to Charles about how ZFS will improve Time Machine performance...
     
CharlesS
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Feb 10, 2008, 01:57 PM
 
I edited my original post to have the log from the time TM took 40 minutes on my machine - check it out.

I suppose that having dual processors probably helps a lot. The problem is that if you're in the middle of doing something on a single-processor machine, your task fights with Time Machine for the same processor, slowing down both your task and Time Machine. And when Time Machine gets slowed down, it takes forever to backup, prolonging the problem. I should probably go bite the bullet and finally buy a new machine... sigh.

Anyway, the point is that with ZFS, it probably wouldn't even be necessary to make all those hard links, and the backups would be much faster even on your machine.

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Feb 10, 2008, 02:13 PM
 
Okay, guys... I will attempt to prove that either HFS+ or Time Machine sucks balls (probably the former).

I have a home directory on my FreeBSD server that has 89353 files. The server is running UFS + softupdates, and I can guarantee that there are many people here that have beefier hardware than my server. I counted them accordingly:

# find . -print | wc -l
89353
I have a Time Machine like script that I run every night that does the following:

rm -rf myhomedirectory.6
mv myhomedirectory.5 myhomedirectory.6
mv myhomedirectory.4 myhomedirectory.5
mv myhomedirectory.3 myhomedirectory.4
mv myhomedirectory.2 myhomedirectory.3
mv myhomedirectory.1 myhomedirectory.2
cd myhomedirectory.0 && find . -print | cpio --quiet -dpl ../myhomedirectory.1
rsync -a --delete /home/me /path/to/backup/myhomedirectory.0/
The key here is the last two lines... Everything above it basically just moves my incremental backup down the line - backup.2 becomes backup.3, etc.

This basically creates a bunch of hard links for files that haven't changed, and complete copies for files that have - this is exactly what Time Machine does. Then, the last line (rsync command) creates a complete archive of the home directory... This becomes backup.0 - what is compared against for the next backup.

This script is actually *less* efficient than Time Machine. For one, there is no FSEvents under FreeBSD. Therefore, the rsync job has to compare my *entire* home directory.

Let's time this script:

# time sh backupscript.sh

real 3m29.267s
user 0m3.371s
sys 0m20.502s
3 minutes, 30 seconds. Now, let's take out the rsync command so that we are just building hard links:

# time `find . -print | cpio --quiet -dpl ../testbackup.1`

real 1m55.887s
user 0m0.424s
sys 0m8.066s
2 minutes.

Like I said, this is an older server, my I/O and CPU is not all that impressive... There is no reason why anyone here with a dual core rig couldn't outperform my server here.
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Feb 10, 2008, 02:20 PM
 
Originally Posted by TETENAL View Post
On my Dual-G5 there is no noticeable slow down. It has 763.039 files on the boot volume. Here's the backup log:

10.02.08 18:36:02 /System/Library/CoreServices/backupd[8834] Backup requested by user
10.02.08 18:36:02 /System/Library/CoreServices/backupd[8834] Starting standard backup
10.02.08 18:36:02 /System/Library/CoreServices/backupd[8834] Backing up to: /Volumes/Backup/Backups.backupdb
10.02.08 18:36:04 /System/Library/CoreServices/backupd[8834] No pre-backup thinning needed: 729.8 MB requested (including padding), 222.36 GB available
10.02.08 18:36:13 /System/Library/CoreServices/backupd[8834] Copied 613 files (3.5 MB) from volume Festplatte.
10.02.08 18:36:13 /System/Library/CoreServices/backupd[8834] Starting post-backup thinning
10.02.08 18:36:18 /System/Library/CoreServices/backupd[8834] Deleted backup /Volumes/Backup/Backups.backupdb/G5/2008-02-09-180552: 222.36 GB now available
10.02.08 18:36:18 /System/Library/CoreServices/backupd[8834] Post-back up thinning complete: 1 expired backups removed
10.02.08 18:36:18 /System/Library/CoreServices/backupd[8834] Backup completed successfully.


So there you have it. One increment took 16 seconds. A regular one is in the order of 2 minutes I would say. I would say I'm content with the performance of HFS+ here.

And yes, disconnecting the backup drive would disrupt the backup obviously. But if you got to go you got to go. Just make a backup when you come back again then. One missed hourly backup is not the end of the world.

This makes me believe quite strongly that CharlesS and I are experiencing FSEvents bugs then, as this is the kind of performance that I would expect. It is definitely *not* the kind of performance we've been getting - nowhere close.

Still, the worst case scenario is that FSEvents folds and the backup has to run against the entire home directory. Those hard links should be created at least as quickly as they were on my server, not within 10 minutes. The only way it would take 10 minutes to create them is if there are serious HFS+ performance bottlenecks.
     
TETENAL
Addicted to MacNN
Join Date: Aug 2004
Location: FFM
Status: Offline
Reply With Quote
Feb 10, 2008, 02:23 PM
 
I disabled one CPU and started a QuickTime movie playback to simulate some system usage. This time the backup took 37 seconds. Not 40 minutes. A better file system would make Time Macine faster for anyone – and if it had block level granularity a great deal more useful as well – but I guess whatever issue you have, it's not due to HFS+.*)

*)
Unless there is some order of magnitude at which HFS+ performance plummets and you are over it and most people aren't. That's possible actually.
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Feb 10, 2008, 02:29 PM
 
Originally Posted by TETENAL View Post
I disabled one CPU and started a QuickTime movie playback to simulate some system usage. This time the backup took 37 seconds. Not 40 minutes. A better file system would make Time Macine faster for anyone – and if it had block level granularity a great deal more useful as well – but I guess whatever issue you have, it's not due to HFS+.*)

*)
Unless there is some order of magnitude at which HFS+ performance plummets and you are over it and most people aren't. That's possible actually.


I agree. I wrote my post before I saw yours. With your results, I think it is safe to say that FSEvents is buggy for some people (such as CharlesS and myself).

However, the one thing that could be concluded from my test, like I said, is that HFS+ is slow at creating hard links. There is no reason why even with a buggy FSEvents that a backup would take any longer than it would to create the entire home directory worth of hard links by scratch (unless backupd also creates a huge CPU load spike or something).
     
CharlesS
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Feb 10, 2008, 02:38 PM
 
Originally Posted by TETENAL View Post
but I guess whatever issue you have, it's not due to HFS+.
Well, the fact that a lot of the time tends to be spent creating hard links, a step that could be completely eliminated if we were using ZFS, tells me that switching to ZFS would shorten the time considerably. Also, mduell's information that HFS+ isn't making true hard links but doing some fakery makes me pretty much convinced that HFS+ is responsible for at least this part of the slowdown.

Besson3c, could you possibly copy that home folder to an HFS+ disk and try the same script on it there, just to see if you get any different results? This would probably be the best way to compare HFS+ vs. another file system directly.

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
besson3c
Clinically Insane
Join Date: Mar 2001
Location: yes
Status: Offline
Reply With Quote
Feb 10, 2008, 02:53 PM
 
Originally Posted by CharlesS View Post
Well, the fact that a lot of the time tends to be spent creating hard links, a step that could be completely eliminated if we were using ZFS, tells me that switching to ZFS would shorten the time considerably. Also, mduell's information that HFS+ isn't making true hard links but doing some fakery makes me pretty much convinced that HFS+ is responsible for at least this part of the slowdown.

Besson3c, could you possibly copy that home folder to an HFS+ disk and try the same script on it there, just to see if you get any different results? This would probably be the best way to compare HFS+ vs. another file system directly.
I might do just that. Shortly after my last post here I also realized that my server also has the overhead of running software RAID-1.

Also, looking at Tetenal's logs, it looks like TM must *copy* the hard link structure rather than recreating it, because in general a copy operation is much faster than writing a bunch of new files one at a time.
     
CharlesS
Posting Junkie
Join Date: Dec 2000
Status: Offline
Reply With Quote
Feb 10, 2008, 03:00 PM
 
Originally Posted by TETENAL View Post
It has 763.039 files on the boot volume.
This also probably has something to do with it - my disk has 1,328,861 files on it, and this is after deleting massive quantities of stuff (which, to be sure, has improved the performance of Time Machine quite a bit - it is now much more common for me to get backups as short as 5-6 minutes, although I'm still not getting results like yours). It was probably close to 2 million before I started on my spring cleaning campaign.

Ticking sound coming from a .pkg package? Don't let the .bom go off! Inspect it first with Pacifist. Macworld - five mice!
     
OreoCookie
Moderator
Join Date: May 2001
Location: Hilbert space
Status: Offline
Reply With Quote
Feb 10, 2008, 05:31 PM
 
Originally Posted by besson3c View Post
I'm pretty sure that HFS+ is not as nice as ext3 either...
ext3 is essentially ext2 + journaling. Kinda like arguing HFS+ and HFS+J are (essentially) different filesystems.
Originally Posted by starman View Post
Oreo, that sounds rather inefficient. Eventually, you'll had a disk that's fragmented to hell.
It seems to be very efficient.
The reason that there is little fragmentation is that the allocation algorithms are very sophisticated. Although I am not an expert, I've read that fragmentation is not an issue with ZFS. It just works, apparently

The other reason is that instructions are grouped together with some knowledge of what will happen: say you're watching a movie and doing something in the background. After some time, ZFS will realize you probably want to watch the whole movie and prefetch a larger part of it.
I don't suffer from insanity, I enjoy every minute of it.
     
 
Thread Tools
 
Forum Links
Forum Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Top
Privacy Policy
All times are GMT -4. The time now is 10:41 PM.
All contents of these forums © 1995-2017 MacNN. All rights reserved.
Branding + Design: www.gesamtbild.com
vBulletin v.3.8.8 © 2000-2017, Jelsoft Enterprises Ltd.,