[DaGuy]

I never used curl before but I'm beginning to feel the need for it given that I'm spending a lot of time in the command line.

Say that I want to download all files in a remote directory using http. Say that files live at "http://acme.com/share/"

What is the curl command for doing this. Feel free to throw in regex and to recommend a good tutorial on curl usage.

Thanks!

[fudo]

I would suggest you start with this tutorial http://curl.haxx.se/docs/httpscripting.shtml.

Also if you have not already done so also look at the man page for curl.

Apparently you can't retrieve directories like you think since HTTP itself does not work in
that fashion. You must have some idea about the content in the directory before you can fetch it.

The following examples will work :
http://www.numericals.com/file[1-100].txt
http://www.numericals.com/file[001-100].txt

But this will not
http://www.numericals.com/file*.txt

I think wildcarding will work with FTP URLs since FTP is capable of handling wildcards.

It maybe worth considering using another scripting language to invoke curl, perhaps
perl, python or even sh. Though this really depends on how often you wish to use curl
against the same URL.


fudo

[int69h]

I'm not sure how to do it with curl, but you can build yourself a copy of wget and recursively grab webpages and their links by running:

wget -r http://www.acme.com/share/index.html


I have no idea why Apple removed wget from the base install. It's quite handy.

[DaGuy]

Great, I will start wget. It offers the simplest approach.

I will explore curl a bit further since the regex capability is attractive and I can see how using it from perl could be a great tool.

Thanks for the info.