Search Engines: Friend or Foe (Pt 2)

Last week we talked about the mechanics of search engines and how they work.  Whenever a search is performed, the search engine will go through its list of indexed sites, fetch and display these sites (or pages) in the form of search listings.  This week, we share more about how search engines decide which pages to list, and in what order to list them.

Search Engine Spiders & Page Indexing

Remember that before a search engine can display your webpage it has to have a record of it.  Search engines send out an automated program known as a spider or a bot, that goes to a website and starts to make a copy of all the content on the site.  Once it has gotten enough information, it brings these results back to the search engine for indexing.  Within this information, there are links to other sites, and the spider will then follow these links to the new websites and index them.  This means that if you place a link to your website from another site that is already in a search engine’s cache (or records), your site will get indexed faster.   Read more about getting your site indexed in Google.

The architecture of your website is very crucial to indexing.  Each trip that a spider makes to your site, it has a limit of how much time to spend there and how much content it can copy.  If your website uses Flash or other non spider-friendly elements, the spider may not be able to index your site.  For this reason, when we design websites, we make sure that the site is both user-friendly, and visible to search engines.

How Search Engines Decide if Your Content is Relevant

Once you have your pages indexed, search engines will display your pages as results when a user searches for a relevant term.  The search engine is a computer and it has no way of reading the content on your page.  What it does however is make use of a mathematical formula ( that takes into account hundreds, if not thousands of factors on your page) to determine a “relevance score”.  All the webpages that meet a basic score are then displayed as results in descending order.  These formulas or algorithms vary from search engine to search engine, and they are closely guarded secrets.  Imagine if people know what factors went into these formulas, they would be able to “beat the system” and probably abuse it for personal gains (hey, we’re only human).

While we don’t know exactly what all these factors, we do make calculated guesses at what some of them are.   All this guesswork eventually evolved into a practice known as Search Engine Optimization (SEO).  It is a proven fact that websites that are optimized for search are more visible to search engines and perform better in search results.

Some SEO Techniques Include:

• Search Engine-Friendly Architecture – Where special programming rules and methods are used to build websites, making it easier for spiders to visit them and collect data
• Backlink Building – Having more websites pointing to yours will increase the chances of your content getting indexed.  Google has also officially mentioned that the number of backlinks to your site is a factor in its algorithm.
• Content Generation – The amount of content on your website and the frequency with it is updated also plays a big part.  Imagine that you are an avid book reader.  Wouldn’t you like to go to a huge store where they stock titles every week compared to small shop that hardly changes its displays?

Overdoing It = Spam

Before you get all psyched up and start posting your links everywhere, most search engines are coming down quite hard on websites that take a shady approach to SEO.  Remember that search engines are all about relevance amd if they start to notice an unnatural growth trend in your website’s backlinks or content generation (especially duplication), you might run the risk of getting your listings removed altogether (Google’s notorious for this).

As with all things online, think about SEO techniques and how they can benefit your users eventually.  As long as you think along those lines, you’ll do fine.

To do great, contact the professionals today

Search Engines: Friend or Foe

There was a rich but snobbish kid in my class who always brought the latest toys to school.  He would only let his appointed friends play with them, so at a young age I experienced the dilemma of selling out or sticking to your beliefs.  Then I grew up and there was Google.

It’s a love-hate situation with the world’s most popular search engine.  if you played by its rules, and played them well, Google (or the other search engines for the matter) could become the most powerful ally in your Internet marketing campaign.  Go all James Dean on it and you’ll find yourself losing precious traffic or in worse scenarios, getting delisted in the search engine results.

How Google Changed the Search Landscape

Prior to 1996, search engines like everything else, ran on the universal fuel of human motivation: Money.  As long as you paid enough money, you were guaranteed of a favorable position in search results listing.  Then Google came along and told the world that they were abolishing paid listings in favor of relevance.  This made sense for Google’s continued survival, since no one would want to use a search engine that delivers only advertisements instead of the content that they were searching for.  Since then Google has been at it’s never-ending  process of refining its search algorithm to deliver faster and more accurate results.

How Search Engines Work

A search engine is like a library, but instead of looking through the shelves for the book you have in mind, you have to go to the librarian and tell her what you are looking for.  She goes through her records to see if the library has a copy of the book and if it does, retrieves it for you.

You might go to the library researching on a particular topic without having a specific book in mind.  In this case the librarian will show you a list of all the related books they have on record.  You then decide which book to choose.

In both cases, the librarian can only offer you a book that exists in her record.  Similarly, a search engine can only display results (websites) that it has on record, or indexed.

Indexing

You’ve written a new book and want to offer a copy of it to the library so that more people can get to read it.

1) Your publisher tells you to wait for a few months, and the library will magically have a record of it (hilarious isn’t he).

2) You leave the book on the library doorstep and hopes that the librarian sees it first thing in the morning.

3) You fill in a submission form and hand it and the book over to the librarian.

Method 3 seems the most probable way of getting your book into the record.  Then again both Methods 2 and 3 depend on the librarian.  Maybe she’s trying to clear the backlog from last week, so your book will have to wait.  Maybe you made an error in the submission form and she cannot process it. Maybe baby.

You notice that the as long as you book is not in the record, no borrower will have access to it.  Now don’t you wish the librarian was a personal friend?

Then Comes the Nightmare of Search Rankings

Your book is finally in the record!  You camp near the counter waiting for a borrower to come and request for your book.  Since it is relatively new and unknown, no one would be asking for it by name (yes, in that same argument no one searches for you on the Internet by your company’s name).  A borrower comes to the counter and asks for books on the topic you have written.  He takes the first 3 books in the records.  Neither of them is the one you wrote.  This happens 10 other times for the rest of the day.

You storm up and demand an explanation from the librarian and she explains that the books on the list are sorted by their popularity and relevance. Great. Now you don’t only have to be an expert in your field, you would have to be popular before anyone even reads your book.  Hey, didn’t you write your book in hope of becoming popular in the first place?!

Just as you are about to give up, a bright light catches your eye..

To Be Continued..

Next Week: Search Engines: Friend or Foe (Pt 2)

Just as the librarian is only human, a search engine is only a program.  Come back same time next week and we’ll show you how to manipulate human emotions and numerical sequences to your own ends.