|
|
How Can I mirror an FTP directory?
|
|
|
|
Fresh-Faced Recruit
Join Date: Apr 2011
Status:
Offline
|
|
Hi,
I work with data that is available online on a couple of FTP sites.
I'd like to "mirror" those sites so that I always have a current copy on my own computer whenever files are added or updated on those sites.
I am wondering if there is any good FTP client, other software package, or other set-up, that can do that automatically.
Thank you.
|
|
|
|
|
|
|
|
|
Clinically Insane
Join Date: Mar 2001
Location: yes
Status:
Offline
|
|
If you are talking straight FTP you can mirror these directories with wget. If these directories are available via SSH you can do the same with rsync.
|
|
|
|
|
|
|
|
|
Fresh-Faced Recruit
Join Date: Apr 2011
Status:
Offline
|
|
Thank you.
Those are new tools to me.
Is there a "how to" you could recommend, or a piece of software that provides and easy-to-use GUI for them?
Thanks, again.
|
|
|
|
|
|
|
|
|
Clinically Insane
Join Date: Mar 2001
Location: yes
Status:
Offline
|
|
I can help you with getting one or both working, I don't know of a GUI.
Do you have SSH available to you on the FTP server?
|
|
|
|
|
|
|
|
|
Fresh-Faced Recruit
Join Date: Apr 2011
Status:
Offline
|
|
Thank you.
I think the answer is "no" about SSH.
It is a public FTP site -- no password required.
|
|
|
|
|
|
|
|
|
Clinically Insane
Join Date: Mar 2001
Location: yes
Status:
Offline
|
|
Okay, well you can compile and install wget yourself using Macports if you are comfortable with this (you'll also need to have XCode installed to do this), or else you can download and install this guy's prepackaged version of wget:
Status-Q � Blog Archive � wget for Mac OS X Leopard
With wget installed, all you have to do is enter the following into your terminal:
wget -r ftp://yourftpsite.com
the -r means "recursive", meaning it will download the contents of any folders it finds. If you have access to Linux many distros will include wget as well.
|
|
|
|
|
|
|
|
|
Fresh-Faced Recruit
Join Date: Apr 2011
Status:
Offline
|
|
Thanks,
Do you know of a way to automate, e.g. to run the comparison/sync once per day or once per week?
|
|
|
|
|
|
|
|
|
Fresh-Faced Recruit
Join Date: Apr 2011
Status:
Offline
|
|
Oh -- I should clarify that it is a BIG dataset -- I want to be sure only to download and replace local files with new files and files that have changed.
|
|
|
|
|
|
|
|
|
Clinically Insane
Join Date: Jun 2001
Location: planning a comeback !
Status:
Offline
|
|
Originally Posted by besson3c
Okay, well you can compile and install wget yourself
Dude, WTF
The OP doesn't even know what SSH and rsync is, do you really think suggesting that he'd compile his own port is useful ?
-t
|
|
|
|
|
|
|
|
|
Clinically Insane
Join Date: Jun 2001
Location: planning a comeback !
Status:
Offline
|
|
|
|
|
|
|
|
|
|
|
Clinically Insane
Join Date: Mar 2001
Location: yes
Status:
Offline
|
|
Originally Posted by burro
Thanks,
Do you know of a way to automate, e.g. to run the comparison/sync once per day or once per week?
If you add the -N option (i.e. wget -rN ftp://yoursite.com) the download will be incremental, only downloading what has changed. As far as automating this, you can do so via a cronjob so that this job runs as often as you'd like.
|
|
|
|
|
|
|
|
|
Clinically Insane
Join Date: Mar 2001
Location: yes
Status:
Offline
|
|
Originally Posted by turtle777
Dude, WTF
The OP doesn't even know what SSH and rsync is, do you really think suggesting that he'd compile his own port is useful ?
-t
Which is why I also provided the link to download the prepackaged wget binary.
If you have a better idea for the OP, share it. It's kind of dumb and annoying that we (collective "we") have to have these incredibly immature little competitions to see who can be the most helpful. This is a brainstorming session, if an idea is not attractive it is not attractive, it's not a big deal. Having options are better than not having any, even if you are only aware of them existing.
|
|
|
|
|
|
|
|
|
Clinically Insane
Join Date: Jun 2001
Location: planning a comeback !
Status:
Offline
|
|
Originally Posted by besson3c
If you have a better idea for the OP, share it.
I did.
-t
|
|
|
|
|
|
|
|
|
Clinically Insane
Join Date: Mar 2001
Location: yes
Status:
Offline
|
|
Originally Posted by turtle777
I did.
-t
Cool!
I hope that one or both of these suggestions are useful to you, original poster...
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Forum Rules
|
|
|
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
|
HTML code is Off
|
|
|
|
|
|
|
|
|
|
|
|