Welcome to the MacNN Forums.

If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

You are here: MacNN Forums > Software - Troubleshooting and Discussion > macOS > Tiger rsync problems

Tiger rsync problems
Thread Tools
wataru
Addicted to MacNN
Join Date: Oct 2001
Location: Yokohama, Japan
Status: Offline
Reply With Quote
Apr 30, 2005, 08:09 PM
 
I am so happy that Tiger's rsync finally understands metadata. In my app iPodBackup I use rsync to backup ones home folder, and the resource-fork aware version of rsync I was using has a bad bug where it can crash and fill up the log file with garbage until the hard drive is full. So now I can ditch this old, buggy verion of rsync, but it's not going very well.

Simply replacing one rsync with the other in the script, and using it to sync to a previously-existing disk image results in errors like the following when trying
Code:
rsync: writefd_unbuffered failed to write 85 bytes: phase "unknown" [receiver]: Broken pipe (32) rsync error: error in rsync protocol data stream (code 12) at /SourceCache/rsync/rsync-20/rsync/io.c(909)
I have no idea why it's crapping out where the other version worked fine.

I thought maybe the new rsync didn't like dealing with data the old rsync made, so I tried making a new disk image from scratch. With that one, the new rsync doesn't give the above error, but it does recopy the metadata (._filename) for every file every time, even if the metadata hasn't changed.

Anyone know why the new rsync behaves so differently? FYI I'm calling it like this:
Code:
/usr/bin/rsync -avuE --exclude-from "exceptions-file.txt" "$HOME"/ "$DESTINATION" --delete
     
krove
Mac Elite
Join Date: Jul 2000
Location: Washington, DC
Status: Offline
Reply With Quote
Jun 2, 2005, 12:53 PM
 
I do not get any such error. (I'm on 10.4.1, BTW). Instead of starting a new Tiger rsync thread, I thought I would continue on this one.

I used a similar command as in the post above (with -avuE options) to backup my home directory to an external HFS+ Journaled firewire disk. The process went fine, except it created thousands of ._metatdata files corresponding to the various files in my home folder.

According to John Siracusa's excellent OS X v10.4 "Tiger" article, he states that these ._files ought only be created when copying to a non-HFS+ formatted disk.

What's the deal? My backup went from 13 GB to 31 GB!!! The drive has plenty of space, but I'd like to know more about the updated rsync version included with OS X and what that -E (for extended attributes) flag really does. Anyone have any clues?

How did it come to this? Goodbye PowerPC. | sensory output
     
wataru  (op)
Addicted to MacNN
Join Date: Oct 2001
Location: Yokohama, Japan
Status: Offline
Reply With Quote
Jun 2, 2005, 12:57 PM
 
I've been able to confirm that the errors I described have something to do with the size (number of files and/or amount of data) in the source directory. I'm very surprised that you aren't seeing this crash given the size of your backup. I can confirm that many others have this problem, as I have received tons of bug reports since updating it to use Apple's rsync.
     
OreoCookie
Moderator
Join Date: May 2001
Location: Hilbert space
Status: Offline
Reply With Quote
Jun 2, 2005, 02:20 PM
 
For me, rsync with Tiger is two things: 1 veeeery slow and 2 it usually aborts with an error.
I don't suffer from insanity, I enjoy every minute of it.
     
wataru  (op)
Addicted to MacNN
Join Date: Oct 2001
Location: Yokohama, Japan
Status: Offline
Reply With Quote
Jun 2, 2005, 08:06 PM
 
If you look in your crashlogs, you'll find that that error is actually a crash.

Incidentally, I reported this to Apple and the bug is still open. So hopefully this will be dealt with.
( Last edited by wataru; Jun 2, 2005 at 08:29 PM. )
     
OreoCookie
Moderator
Join Date: May 2001
Location: Hilbert space
Status: Offline
Reply With Quote
Jun 3, 2005, 05:12 AM
 
It's a known issue, I was looking into it when i ran into it. Tons of complaints, no solution.

I hope Apple will eventually include rdiff, an rsync based backup app.
I don't suffer from insanity, I enjoy every minute of it.
     
krove
Mac Elite
Join Date: Jul 2000
Location: Washington, DC
Status: Offline
Reply With Quote
Jun 19, 2005, 12:09 PM
 
Let's hope 10.4.2 fixes this.

How did it come to this? Goodbye PowerPC. | sensory output
     
Alexander_Lamb
Fresh-Faced Recruit
Join Date: Jul 2005
Location: Geneva, Switzerland
Status: Offline
Reply With Quote
Jul 27, 2005, 08:42 AM
 
Still problems with rsync with the latest version of Tiger.
I discovered that it crashes more systematically when writing to a mounted drive (local or not) than when using rsh which is usually ok.

Otherwise, everything gets REALLY SLOW when using rsync. I am suspecting two things here:

1) Spotlight doing something stupid (like reindexing the complete backup destination)

2) Backuping too much.

The point 2 is a problem as I discovered that not only all the ressource forks were copied systematically (even from HFS+ to HFS+) but that some files which were not modified get copied (like my complete iPhoto library).

Summary:

WIth the crashing, the speed and the fact it copies way too many files.... rsync is unusable on Tiger at this point.
Alexander Lamb
Geneva Switzerland
     
osxrules
Mac Enthusiast
Join Date: Oct 2004
Status: Offline
Reply With Quote
Aug 2, 2005, 01:05 PM
 
I use SuperDuper. Try versiontracker for rsyncX - there's supposed to be problems with the built-in version in Tiger.
     
wataru  (op)
Addicted to MacNN
Join Date: Oct 2001
Location: Yokohama, Japan
Status: Offline
Reply With Quote
Aug 2, 2005, 01:24 PM
 
Originally Posted by osxrules
there's supposed to be problems with the built-in version in Tiger.
Gee, really?

SuperDuper is not a replacement for rsync, and rsyncX has its own problems, namely no support for ACLs and a tendency to go crazy and fill hard drives with garbage log files. The exploding log thing is a known bug and yet the developer hasn't done anything about it for almost a year. There are good reasons for wanting Apple's rsync to Just Work™.
     
TigerKR
Fresh-Faced Recruit
Join Date: Jan 2004
Location: Middlesex County, MA, US
Status: Offline
Reply With Quote
Sep 9, 2005, 09:33 PM
 
I get this error whenever I use the -E flag in 10.4.2:

macuser $ sudo rsync -aEv --delete /Users/macuser/ /Volumes/Backup/macuser/
building file list ... done
Bus error
rsync: writefd_unbuffered failed to write 85 bytes: phase "unknown" [receiver]: Broken pipe (32)
rsync error: error in rsync protocol data stream (code 12) at /SourceCache/rsync/rsync-20/rsync/io.c(909)
     
wataru  (op)
Addicted to MacNN
Join Date: Oct 2001
Location: Yokohama, Japan
Status: Offline
Reply With Quote
Sep 10, 2005, 04:30 AM
 
Yes, that's one of the two typical errors that occur for no reason on Tiger. The solution is to wait for Apple to fix rsync, or to use the version of rsync that's bundled with RsyncX, keeping in mind that it has horrible bugs of its own.
     
blue80
Fresh-Faced Recruit
Join Date: Aug 2007
Status: Offline
Reply With Quote
Aug 30, 2007, 07:14 AM
 
After many hours and much fiddling trying to get backup happening. Rsync Lart 2.6.6 Darwinports version finally works for me on Tiger.

rsync-lart version 2.6.6 for Mac OS X

Not sure how good it is with resource forks but at least it can do a backup without the interminable broken pipe error.

Well as good as it can....see Backup Bouncer for independent testing of capability:
n8gray.org: Introducing Backup-Bouncer
     
Peter
Addicted to MacNN
Join Date: Oct 2002
Location: England | San Francisco
Status: Offline
Reply With Quote
Aug 30, 2007, 09:29 AM
 
please dont resurrect super old threads.
I use
rsync version 2.6.3 protocol version 28
and it works fine - I *thought* all rsync bugs (within reason!) were fixed in 10.4.9 ?
Primarily the bug with meta date (-E)
we don't have time to stop for gas
     
blue80
Fresh-Faced Recruit
Join Date: Aug 2007
Status: Offline
Reply With Quote
Sep 2, 2007, 12:23 AM
 
Originally Posted by Peter View Post
please dont resurrect super old threads.
I use
2.6.3 protocol 28
and it works fine - I *thought* all rsync bugs (within reason!) were fixed in 10.4.9 ?
Primarily the bug with meta date (-E)
Sorry, but bugs still persist and I couldn't find any newer threads here.

Backup bouncer still records metadata problems with latest version for 10.4.10 - same as yours:
n8gray.org: Backup Bouncer

I couldn't get it to work with a mounted remote volume, but that was fixed with patch to rsync (rsync-lart) version 2.6.6 protocol version 29.
     
   
 
Forum Links
Forum Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Top
Privacy Policy
All times are GMT -4. The time now is 10:50 PM.
All contents of these forums © 1995-2017 MacNN. All rights reserved.
Branding + Design: www.gesamtbild.com
vBulletin v.3.8.8 © 2000-2017, Jelsoft Enterprises Ltd.,