 |
 |
How can I download an entire directory from the web?
|
|
|
|
|
Senior User
Join Date: Nov 2003
Status:
Offline
|
|
I would like to download everything that is in a directory on a website.
For example: the images folder... in www.com: the web starts here... there are 100s of photos. I would like to be able to just download everything to a folder on my computer.
Do I need to use cURL for this? I've never used it before, so I am not sure how it works.
|
|
|
| |
|
|
|
|
|
|
|
Mac Elite
Join Date: Mar 2007
Location: WI, United States
Status:
Offline
|
|
I'm no expert, but couldn't Automator do this?
|
|
|
| |
|
|
|
|
|
|
|
Senior User
Join Date: Nov 2003
Status:
Offline
|
|
That might be another option. I have never used automator either. Does anyone use Automator frequently? It seems pretty useless to me.
|
|
|
| |
|
|
|
|
|
|
|
Addicted to MacNN
Join Date: Feb 2003
Location: NY²
Status:
Offline
|
|
Automator could do this, and it's far from useless.
Another way, which will be a little easier, would be to use an app like SiteSucker to download the images folder.
|
|
|
| |
|
|
|
|
|
|
|
Moderator  Join Date: Dec 2000
Location: Polwaristan
Status:
Offline
|
|
Originally Posted by mdc 
Automator could do this, and it's far from useless.
Another way, which will be a little easier, would be to use an app like SiteSucker to download the images folder.
Seconded. SiteSucker is a good app.
|
|
|
| |
|
|
|
|
|
|
|
Addicted to MacNN
Join Date: Aug 2004
Location: FFM
Status:
Offline
|
|
|
|
|
|
| |
|
|
|
|
|
|
|
Mac Elite
Join Date: Mar 2003
Location: Virginia
Status:
Offline
|
|
You can try WGet which is a generic command line program that downloads a URL to a specified level. There are f ront-ends to WGet on download sites to make it easy to use.
I also recommend DownThemAll which is a Firefox plugin that does exactly what you need.
|
|
|
| |
|
|
|
|
|
|
|
Clinically Insane
Join Date: Mar 2001
Location: yes
Status:
Offline
|
|
Unix wget. There are several OS X GUIs for this, I believe, including SiteSucker (AFAIK), a Cocoa tool, and from the looks of it, some others:
wget os x gui - Google Search
|
|
|
| |
|
|
|
|
|
|
|
Mac Elite
Join Date: Nov 2003
Location: The back of the room
Status:
Offline
|
|
That doesn't appear to be an open directory. Without any URIs, would these apps be able to grab everything at once?
|
|
|
| |
|
|
|
|
|
|
|
Clinically Insane
Join Date: Mar 2001
Location: yes
Status:
Offline
|
|
wget spiders through a site from a starting URL just like a search engine does. It respects the site's robots.txt instructions, although there is a command line argument to ignore these files. Wget also requires the -r flag to operate recursively through the entire website.
I don't know anything about the wget GUIs... wget is pretty simple to use though.
|
|
|
| |
|
|
|
|
|
|
|
Mac Elite
Join Date: Nov 2003
Location: The back of the room
Status:
Offline
|
|
That's what I'm getting at. Without a page full of URIs a tool like this is useless for the task at hand. And from what I saw of the OP's link (a redirect to the server root) there is no index page like that for that directory.
Though if there is, you're in business.
|
|
|
| |
|
|
|
|
|
|
|
Clinically Insane
Join Date: Mar 2001
Location: yes
Status:
Offline
|
|
Oh, I thought the provided URL was just a dummy, example URL...
|
|
|
| |
|
|
|
|
|
|
|
Mac Elite
Join Date: Jan 2006
Status:
Offline
|
|
|
|
|
|
| |
|
|
|
|
|
|
|
Mac Elite
Join Date: Nov 2003
Location: The back of the room
Status:
Offline
|
|
Originally Posted by besson3c
Oh, I thought the provided URL was just a dummy, example URL...
You're probably right. But my point still stands.
|
|
|
| |
|
|
|
|
|
|
|
Clinically Insane
Join Date: Mar 2001
Location: yes
Status:
Offline
|
|
Originally Posted by zro
You're probably right. But my point still stands.
Unless Apache's dir_module is enabled, which is provided and enabled with the stock Apache by default, I believe.
|
|
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
 
|
|
|
Forum Rules
|
|
|
|
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
|
HTML code is Off
|
|
|
|
|
|
|
|
|
|
|
|
|
Top
