 |
 |
Does the Yahoo bot slam your site?
|
|
|
|
|
Clinically Insane
Join Date: Oct 2001
Location: San Diego, CA, USA
Status:
Offline
|
|
Looking through our usage statistics, I see Yahoo's Slurp bot hits my site about as much as all the other spiders put together. The second highest is the Google bot, with 1/4 as many hits. I was just wondering if anybody else is seeing similar numbers.
|
|
Chuck
___
"Instead of either 'multi-talented' or 'multitalented' use 'bisexual'."
|
| |
|
|
|
|
|
|
|
Mac Elite
Join Date: Nov 2003
Location: The back of the room
Status:
Offline
|
|
I get more msnbot hits with slurp number two and googlebot a fairly distant third.
|
|
|
| |
|
|
|
|
|
|
|
Fresh-Faced Recruit
Join Date: Apr 2007
Status:
Offline
|
|
use a robots.txt file. it will help.
|
|
|
| |
|
|
|
|
|
|
|
Mac Elite
Join Date: Nov 2003
Location: The back of the room
Status:
Offline
|
|
What, something like...
"Dear msnbot,
I'd like to take this opportunity to ask you to reduce the number of times you crawl my files and directories. Pretty please?
Yours,
zro"
|
|
|
| |
|
|
|
|
|
|
|
Dedicated MacNNer
Join Date: Mar 2001
Location: Indianapolis, IN USA
Status:
Offline
|
|
|
|
|
|
| |
|
|
|
|
|
|
|
Mac Elite
Join Date: Nov 2003
Location: The back of the room
Status:
Offline
|
|
That doesn't address the issue. It's not that you want them to stop, just to tone it down. A robots.txt file can't do that. It's either index or not.
|
|
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
 
|
|
|
Forum Rules
|
|
|
|
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
|
HTML code is Off
|
|
|
|
|
|
|
|
|
|
|
|
Top