Welcome to the MacNN Forums.

If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

You are here: MacNN Forums > Software - Troubleshooting and Discussion > Developer Center > stopping google

stopping google
Thread Tools
Professional Poster
Join Date: Jan 2000
Location: south
Status: Offline
Reply With Quote
Mar 22, 2003, 11:30 PM
 
Is there anyway to stop googlebots from indexing your site etc?
     
Mac Enthusiast
Join Date: Jul 2001
Location: Silicon Valley
Status: Offline
Reply With Quote
Mar 23, 2003, 12:13 PM
 
I think Google's Bots will respect the 'robots text' tags.
Just make sure you have those tags in your site.
Heck you could probably e-mail them and ask how you can avoid having your site Googled.
Reality is the playground of the unimaginative
     
Clinically Insane
Join Date: Nov 1999
Status: Offline
Reply With Quote
Mar 23, 2003, 04:05 PM
 
http://www.robotstxt.org

Google respects robots.txt. Unfortunately, many spambots don't, and those may be what you're really after.
You are in Soviet Russia. It is dark. Grue is likely to be eaten by YOU!
     
Mac Elite
Join Date: Mar 2001
Status: Offline
Reply With Quote
Mar 24, 2003, 09:49 PM
 
Originally posted by OSX Abuser:
I think Google's Bots will respect the 'robots text' tags.
Just make sure you have those tags in your site.
Heck you could probably e-mail them and ask how you can avoid having your site Googled.
It's robots.txt, not tags. Well, you could put tags in your pages for robots to not index.

I'd do both.

Remember, bots don't index your site voluntairily... they don't have to respect the robots.txt file..
     
Grizzled Veteran
Join Date: Jun 2001
Location: Melbourne, Australia
Status: Offline
Reply With Quote
Mar 25, 2003, 04:01 AM
 
Originally posted by Raman:
It's robots.txt, not tags. Well, you could put tags in your pages for robots to not index.

I'd do both.

Remember, bots don't index your site voluntairily... they don't have to respect the robots.txt file..
And if you're really annoyed with them, you could modify your apache config to deny all requests from certain user agents with a string pattern matching the name of the particular robot(s) you wish to hide from. But that's a pretty heavy measure, only to be used if all else fails.
Computer thez nohhh...
     
   
Thread Tools
Forum Links
Forum Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Top
Privacy Policy
All times are GMT -5. The time now is 08:55 PM.
All contents of these forums © 1995-2011 MacNN. All rights reserved.
Branding + Design: www.gesamtbild.com
vBulletin v.3.8.7 © 2000-2011, Jelsoft Enterprises Ltd., Content Relevant URLs by vBSEO 3.3.2