How do Search Engines work?

Search Engines don’t automatically know about every single website and webpage ever created. They must find and index websites and web pages. Search engines do this by sending out spiders, crawlers or robots that move from webpage to webpage using links found on those web pages. Links show spiders the way from one webpage to another webpage or website. Once these spiders find and index websites and web pages the search engine must then determine a way to rank them in order of importance. The ranking is done by an algorithm the search engine programs. Algorithms are highly guarded secrets by search engines. An algorithm is the secret recipe that a search engine uses to make itself better than other competing search engines. The ultimate goal for a search engine is to provide the most relevant result, the first time and every time, based on the search a person conducts.

Let’s describe this in another way. Let’s say you went to the first website you knew about. From there you began to copy and paste every single webpage you could find on that website into a folder on your computer. After you finished copying and pasting every webpage, you noticed there was a link that would take you to another website, so you clicked on it and followed it to the other website. Once you arrived at the new website you repeated the process of copying and pasting every webpage into a folder on your computer before clicking on a link to another website. You continue to repeat the entire process over and over again until you arrive at a website that leads you to a dead end with no more links to follow. So now you have a folder filled with millions of websites that you must somehow organize by topic and order of importance. Here comes your first test. A friend of yours is visiting Chicago and she asks you where she can find the best deal on a new Louis Vuitton handbag in the city. Your job is now to provide her with the best website from the millions of websites spanning millions of topics in your folder to give her the information she is seeking. And by the way, you have less than three seconds to complete the task. This is the challenge a search engine faces millions of times each day.

Back to SEM FAQ