Last week we talked about the mechanics of search engines and how they work.  Whenever a search is performed, the search engine will go through its list of indexed sites, fetch and display these sites (or pages) in the form of search listings.  This week, we share more about how search engines decide which pages to list, and in what order to list them.

Search Engine Spiders & Page Indexing

Remember that before a search engine can display your webpage it has to have a record of it.  Search engines send out an automated program known as a spider or a bot, that goes to a website and starts to make a copy of all the content on the site.  Once it has gotten enough information, it brings these results back to the search engine for indexing.  Within this information, there are links to other sites, and the spider will then follow these links to the new websites and index them.  This means that if you place a link to your website from another site that is already in a search engine’s cache (or records), your site will get indexed faster.   Read more about getting your site indexed in Google.

The architecture of your website is very crucial to indexing.  Each trip that a spider makes to your site, it has a limit of how much time to spend there and how much content it can copy.  If your website uses Flash or other non spider-friendly elements, the spider may not be able to index your site.  For this reason, when we design websites, we make sure that the site is both user-friendly, and visible to search engines.

How Search Engines Decide if Your Content is Relevant

Once you have your pages indexed, search engines will display your pages as results when a user searches for a relevant term.  The search engine is a computer and it has no way of reading the content on your page.  What it does however is make use of a mathematical formula ( that takes into account hundreds, if not thousands of factors on your page) to determine a “relevance score”.  All the webpages that meet a basic score are then displayed as results in descending order.  These formulas or algorithms vary from search engine to search engine, and they are closely guarded secrets.  Imagine if people know what factors went into these formulas, they would be able to “beat the system” and probably abuse it for personal gains (hey, we’re only human).

While we don’t know exactly what all these factors, we do make calculated guesses at what some of them are.   All this guesswork eventually evolved into a practice known as Search Engine Optimization (SEO).  It is a proven fact that websites that are optimized for search are more visible to search engines and perform better in search results.

Some SEO Techniques Include:

• Search Engine-Friendly Architecture – Where special programming rules and methods are used to build websites, making it easier for spiders to visit them and collect data
• Backlink Building – Having more websites pointing to yours will increase the chances of your content getting indexed.  Google has also officially mentioned that the number of backlinks to your site is a factor in its algorithm.
• Content Generation – The amount of content on your website and the frequency with it is updated also plays a big part.  Imagine that you are an avid book reader.  Wouldn’t you like to go to a huge store where they stock titles every week compared to small shop that hardly changes its displays?

Overdoing It = Spam

Before you get all psyched up and start posting your links everywhere, most search engines are coming down quite hard on websites that take a shady approach to SEO.  Remember that search engines are all about relevance amd if they start to notice an unnatural growth trend in your website’s backlinks or content generation (especially duplication), you might run the risk of getting your listings removed altogether (Google’s notorious for this).

As with all things online, think about SEO techniques and how they can benefit your users eventually.  As long as you think along those lines, you’ll do fine.

To do great, contact the professionals today