SEO: Getting your site indexed

Location: Articles > SEO > Here
Written By: Site Admin
Date: 11 Jul 2005
Level: 2 (Novice)

A great site needs visitors whether its purpose is to inform, educate, entertain, generate sales, to earn revenue from advertising or for the sheer joy of having a successful web site however your target audience needs to know about your web site and where better to start than search engines. The majority of sites get found through search engines and I am yet to find someone who doesn't use the Internet without a search engine for the reason that it is similar to driving in a city you have never been to without a map. Search engines direct users to where they want to go to find what they want conveniently and that is what you want them to do with the end result being that they arrive at your site. This article will tell you the basics about getting indexed by the major search engines and directories as well as how to get started. A lot of the readers of this article request more information on Google as that seems to be the most popular search engine. There is a section for Google specific information.

Table of Contents

  1. How indexing works
  2. How to get your site indexed
  3. Reasons for being indexed
  4. Why aren't I indexed yet?
  5. List of search engines and indexes

How indexing works

First of all, I will introduce you to bots. To some they will find this easy to understand however it is a key component that should be understood by anyone taking SEO seriously. A bot (short for robot) performs actions without the need for human assistance, for example, enemy units in a game are bots, the automatic messages in IM clients and so on. Much like these a search engine's bot accesses web sites across the web and analyses the source code following links, this process is called crawling and enables hundreds of millions of pages to be indexed. The count for Google's list of web pages is actually in billions and that figure increases rapidly with more awareness of the Internet and web sites become easier to create and put on the web.

How to get your site indexed

Chances are if you have been working on your site for a few weeks or more, you are probably indexed already, search for your site title or a phrase from your home page to check if you are. If you are not listed, then there are two methods, the first is to put your web site link onto a directory or do a link exchange with another listed page and wait a few days for a bot to re-crawl the site with your link on it. The benefit to doing this is you will gradually get listed to many search indexes as more bots for other search engines "find" your site. The other method is manually listing your site at each search engine and a list of these along with links to the actual "add your site" form.

Reasons for being indexed

Search engines account for over 50% of traffic on the Internet. There are hundreds of millions of search queries submitted every day. You may already know this and be an avid searcher yourself, it provides a map to find the answer to almost any question or the resource containing information on a topic you are researching. It is evident that a high search engine ranking will mean that your site will be on that list and it will be your site getting the visitors. More visitors means a greater community if you have discussion forums, more sales, more advertising revenue or perhaps the sheer joy of having a successful site where you can share all your information with the rest of the world!

Why aren't I indexed yet?

It takes time; bots may not frequent the site containing your web site link because the page doesn't update much so you may need to wait a week or two. Another problem could be that the specific page with your link is not crawled, possible because it isn't linked to properly from another part of the site, it is a dynamic link or it may be classed accidentally or rightfully as a link farm (a page with no intention other than to link to many other sites with no content regarding the description - click on the term to see a full definition). Of course it could be your site causing the problem; it could be breaking search engine guidelines that the bot identifies.

List of search engines and indexes

This is a list of my top 5 search engines

Google
http://www.google.com
The cream of the crop. The most popular search engine and voted best by many without doubt.
Add URL: http://www.google.com.au/addurl

Yahoo!
http://www.yahoo.com
Not far behind Google, Yahoo uses a similar technique to Google.
Add URL: http://submit.search.yahoo.com/free/request

MSN
http://search.msn.com
Another competitor, again, it uses a similar technique to MSN and Yahoo!
Add URL: combined with Yahoo!

Altavista
http://www.altavista.com
Remember? This used to be my favorite search engine and combined with Babelfish, it was great!
Add URL: combined with Yahoo!

Teoma
http://www.teoma.com
Results are used for AskJeeves so Teoma is regarded important in that sense. It also has some nice features. Crawler indexing only, meaning there is no "Add URL" form.

Also, have a look at our list of directories offering free submission.

There are hundreds of other search engines and if you want to find a larger, Google search: search engine list

AllTheWeb

Hotbot