[passmaster16]

Well for as long as I've dabbled with web design, i still don't know of a sure way of getting my site added to any of the major search engines (yahoo, google, etc) I've tried to submit the address of my site to their database on numerous occasions with no luck. Is there really no way around paying one of these companies to list it for me or is there something that I am just missing?

I can do a search on my site and drill down 10 pages with no result. I don't get it...

[waxcrash]

Basically, every search engine gets info from DMOZ.

http://dmoz.org/

[Mediaman_12]

Getting your site listed on ANY search engine is HARD.
It is NOT just a matter of submitting your site every so often. Getting good rankings relies on a combination of a wide range of things ranging from the way you code your HTML pages to how many and what sites link to you.

the Site Point Forums have a section devoted to search engine optimisation and a entire forum just dedicated to getting Google rankings.

[PurpleGiant]

There is a lot of factors, and the following is oversimplified, but gives the general idea:

1) List you site well in dmoz.org - then wait a few months
2) Make sure important words are listed in the page title, in a heading at the top of the page, and in the first few sentences.
3) Be popular! Lots of places linking to you all over the net will increase your ranking a lot. (not dodgy link farms - legitimate sites with high rankings themselves are the ones that help)
4) Code well. Well coded sites get higher rankings.
5) Update regularly

Obviously you don't need to do every single thing above to get listed (eg some sites are super popular but poorly coded), however by following the above you will get listed eventually.

[King Bob On The Cob]

Originally posted by Mediaman_12:
Getting your site listed on ANY search engine is HARD.
It is NOT just a matter of submitting your site every so often. Getting good rankings relies on a combination of a wide range of things ranging from the way you code your HTML pages to how many and what sites link to you.

the Site Point Forums have a section devoted to search engine optimisation and a entire forum just dedicated to getting Google rankings.

I've found it's usually a matter of time before Google finds your site, even if you think it's not listed.

[CMYKid]

as was mentioned a bit before, don't waste time an' money on companies purporting to submit you. they do little you can't accomplish on your own. don't, however, expect to rely much on coding in terms that are hit often. search engines have moved past that for the most part, else everyone would just put 'Britney Spears nude' in their tags. like the guy said befo', one of the largest determining factors is site content and linking, so if you have an assload of info and link to and are linked to everyone then you'll have a good platform from which to build.

also, you didnt say what the aim of yoru site was exactly, but you of course shouldnt overlook buying traffic if you're wanting to accomplish some commerce. we presently spend a minimum of $5000 a day for a couple of our sites, paying for keyword link rankings from Google and Overture. obviously you might not need quite that volume but it's still fairly affordable to do in lesser quantities.

[Demonhood]

use a robots.txt file.

[Chris O'Brien]

Originally posted by Demonhood:
use a robots.txt file.

Isn't a robots.txt file use for excluding robots from a site/parts of a site?

I set my G3 up as a web server using a no-ip.com address and within 3 days the google bot found it. Searching for the site name came up as the top hit on google, but searching for content wasn't so good...

I may have been just lucky, but I like to think it says a lot for a well coded site .

[Millennium]

Originally posted by Black Book:
Isn't a robots.txt file use for excluding robots from a site/parts of a site?

It can be used for such, yes, but it can also be used to deny only some robots but allow others. It would be useful for denying spambots, but most of these don't respect robots.txt anyway.

In any case, though, if you use a robots.txt file but let search engines through, many of them will reward you for doing so.

[passmaster16]

I appreciate the responses so far. From the little research I've done so far, I would gather that the DMOZ project may not always be as fair since human beings make the decision if the site gets added to the index or not. Now I'm sure that there are many DMOZ editors that are fair and perform their duties based on the DMOZ regulations but I see that system as being a roadblock. I've submitted to DMOZ once and nothing ever happened. Apparently my URL got rejected. Although I have read that for some categories the backlog is just too much for the editors to handle.

I am surprised for having my site up as long as it has been that google hasn't found it automatically yet. It probably has something to do with my poor coding.

Anyway, I'm interested in the robots.txt file although it seems like as the other poster mentioned that it is used for exclusions...so if it is just used for excluding certain areas of a site from indexing, how does it help with my problem of getting in the index in the first place?

[djohnson]

Originally posted by waxcrash:
Basically, every search engine gets info from DMOZ.

http://dmoz.org/

Not true. I have sites that are ranked very well yet do not show in DMOZ.

You want to get ranked fast by Google or one of the other engines? Make sure that a page that is indexed has a link to your page. Oh and your Signature on this board will work as well!

[Demonhood]

when you have a robots.txt file, the spiders detect it. the presence of this file alone seems to help them index your site. for example, here is apple's:

Code:
# robots.txt for http://www.apple.com/
User-agent: *
Disallow:

this disallows nothing. if they had used 'Disallow: /' that would have disallowed everything. so use apple's example above, have friends visit your page (especially if they have google's browser toolbar in IE), and make sure you have plenty of text. my site wouldn't be ranked at all if i didn't have smartass photo captions and a quotes page. so one day you too may be highly ranked for 'amish transsexual' like mine is.

[Mastrap]

Google loves frequently updated content. Since I added a blog to my portfolio site it went from nowhere to page one on google for relevant searches.