[burro]

Hi,

I work with data that is available online on a couple of FTP sites.

I'd like to "mirror" those sites so that I always have a current copy on my own computer whenever files are added or updated on those sites.

I am wondering if there is any good FTP client, other software package, or other set-up, that can do that automatically.

Thank you.

[besson3c]

If you are talking straight FTP you can mirror these directories with wget. If these directories are available via SSH you can do the same with rsync.

[burro]

Thank you.

Those are new tools to me.

Is there a "how to" you could recommend, or a piece of software that provides and easy-to-use GUI for them?

Thanks, again.

[besson3c]

I can help you with getting one or both working, I don't know of a GUI.

Do you have SSH available to you on the FTP server?

[burro]

Thank you.

I think the answer is "no" about SSH.

It is a public FTP site -- no password required.

[besson3c]

Okay, well you can compile and install wget yourself using Macports if you are comfortable with this (you'll also need to have XCode installed to do this), or else you can download and install this guy's prepackaged version of wget:

Status-Q � Blog Archive � wget for Mac OS X Leopard

With wget installed, all you have to do is enter the following into your terminal:

wget -r ftp://yourftpsite.com

the -r means "recursive", meaning it will download the contents of any folders it finds. If you have access to Linux many distros will include wget as well.

[burro]

Thanks,

Do you know of a way to automate, e.g. to run the comparison/sync once per day or once per week?

[burro]

Oh -- I should clarify that it is a BIG dataset -- I want to be sure only to download and replace local files with new files and files that have changed.

[turtle777]

Originally Posted by besson3c 

Okay, well you can compile and install wget yourself

Dude, WTF

The OP doesn't even know what SSH and rsync is, do you really think suggesting that he'd compile his own port is useful ?

-t

[turtle777]

Originally Posted by burro 

I am wondering if there is any good FTP client, other software package, or other set-up, that can do that automatically.

I have never used that functionality, but have a look at those, they both support synchronizations (I'm not sure to what extent automated):

FTP Client Software to Upload and Download Files

FTP, SFTP, WebDAV, Cloud Files, Google Docs & Amazon S3 Browser for Mac & Windows.

-t

[besson3c]

Originally Posted by burro 

Thanks,

Do you know of a way to automate, e.g. to run the comparison/sync once per day or once per week?

If you add the -N option (i.e. wget -rN ftp://yoursite.com) the download will be incremental, only downloading what has changed. As far as automating this, you can do so via a cronjob so that this job runs as often as you'd like.

[besson3c]

Originally Posted by turtle777 

Dude, WTF

The OP doesn't even know what SSH and rsync is, do you really think suggesting that he'd compile his own port is useful ?

-t

Which is why I also provided the link to download the prepackaged wget binary.

If you have a better idea for the OP, share it. It's kind of dumb and annoying that we (collective "we") have to have these incredibly immature little competitions to see who can be the most helpful. This is a brainstorming session, if an idea is not attractive it is not attractive, it's not a big deal. Having options are better than not having any, even if you are only aware of them existing.

[turtle777]

Originally Posted by besson3c 

If you have a better idea for the OP, share it.

I did.

-t

[besson3c]

Originally Posted by turtle777 

I did.

-t

Cool!

I hope that one or both of these suggestions are useful to you, original poster...