Chapter 2: Search Engines: Crawling, Indexing, and Ranking
Welcome to the second chapter of IMG’s SEO 101 course! In Chapter 1, we addressed how SEO is very much a question and answers game – with search engines acting as a sort of super-smart magic 8 ball. This time, we’ll be diving deeper into search engines and their functionality.
How Do Search Engines Work?
Search engines like Google and Bing aggregate all the billions of potential links throughout the internet and sort them in a way that is most useful to the individuals searching. They’re able to do this in seconds or less. Such a feat requires a massive amount of computational ability – but with so many similar web pages search engines rely on a three-pronged system for organizing everything long before someone enters in a search.
Crawling: Analyze the entire internet for content,
taking both content and code into account for each web page they crawl.
Indexing: Once analyzed, store web pages found during the crawling process for future use. Once indexed, a page is officially able to come up in search results.
Ranking: Organize all indexed web pages by how relevant they are to the particular topic or search – utilizing keywords and phrases. The higher the rank – the earlier in the search results a web page appears.
What Is Web Crawling?
Put simply, crawling is the process in which search engines discover and analyze all of the content that exists on the internet utilizing robot software. These bots are commonly called spiders or crawlers – and work very much how the name implies. Leaning on the original internet metaphor, the crawl bots quite literally crawl across the invisible “web” of code behind the internet to read everything.
What Is Search Indexing?
Whether it be an article, image, video, or even a download file, crawl bots document everything they come into contact with. In that way, they act like digital cartographers – being sent out by search engines to map out every new link that pops up. Search engines process and store that in an index, a huge digital archive of all the content and web pages they’ve ever discovered and determine is worth saving.
Search Engine Ranking
Archiving all the content on the web is all well and good, but with so much out there, proper organization is critical for search engines to provide value. This ordering of search results by relevance is known as ranking. Utilizing keywords, search engines assign every web page a numbered rank for each of the potential keywords and search terms that can apply to it. In general, you can assume that the higher a website is ranked, the more relevant the search engine believes that site is to the search. When someone searches something on Google or Bing, the search engine scours its repository of links and displays them based on the most recent rankings and the keywords used in the search.
Rankings, however, are not set in stone – and in some cases can fluctuate frequently. This is where the competition comes into play. If you want your website to show up on Google in Bing – you have to make sure your site can be properly crawled, and that the content you place on it is highly valuable to your audience. But like we said, rankings are not set in stone – once you start to rank high, you have to actively fight to stay there.
Your Next Steps
Now we have a better understanding of how search engines function. In Chapter 3, we’ll discuss how your website affects SEO and how to ensure your site is easily accessible by search engines for crawling.
We encourage you to keep up with this ongoing series for even more in-depth breakdowns of how SEO works, and the best practices we employ to help our clients grow. In Chapter 3, we’ll discuss keywords and the role they play in
If you have any questions about SEO that you need answers to now or are considering bringing in some expert help, please reach out! The IMG team is always here to help you and your business grow.
Sign up for our newsletter to stay up to date on new SEO Series releases:
