For most of us, search engine optimization (SEO) can be more than a little confusing. The main idea is to increase traffic to your site through optimizing pages for search engines, but how does it work? In plain English, how do you get the pages within your site “read” by the search engines and stored in the database of existing pages? In SEO terms, how do you get your site crawled and indexed?

Search engines use two major areas of assessment to produce search engine result pages (SERPs) most relevant to a specific search: document analysis and link analysis.

Document Analysis

Document analysis comprises several factors. The search engine crawls pages for keywords related to the search while taking both quantity and location into account. Keywords in more important places carry more weight.  For example, a search engine assumes keywords in your domain name, title tag and H1 tags (headings) as more likely to convey subject matter than page content and captions.  Search engines also surmise that pages with multiple instances of a keyword are more relevant to the query, the word or phrase entered in a search engine.

Actual page content is another factor that search engines are able to recognize and use to rank pages. Search engines use semantics and lexical analysis to “read” text and judge its quality. Other factors are also measured, such as how long a viewer spends on your page.  How do you keep a viewer on your page longer? You can compel them to stay with unique, intelligent content.

Link Analysis

Search engines not only strive to produce relevant SERPs, but also quality SERPs, a process which is achieved through link analysis.  Search engines assume that the more sites there are that link to your site (called backlinking), the more authoritative your site is.  Popularity equals importance. 

Search engines also “read” what the backlinking site says about your site. The anchor text (the actual text being linked) and the text directly surrounding the link are both considered. If the anchor text for the backlink is “this is a terrible site,” search engines take that text into account. Therefore, you wouldn’t want another site to link to yours with the anchor text “click here,” but something more relevant, such as your company name or descriptive keywords. 

Links from any old site won’t do either. The more trusted or authoritative the site is that backlinks to your site, the more weight that backlink will carry. This being said, backlinks from poorly coded sites with inferior content can actually hurt your ranking. Link farms, sites that exist solely to house links for the purpose of influencing rank, are an example of this type of harmful spam. Sites that participate in link farms are penalized by search engines and given a lower ranking. 

Quality Website

Most importantly, for your site to be crawled and properly indexed, you need a site worthy of traffic. This means having a site with good usability, professional design and high-quality content. A site with good usability is easily crawled because it has clear navigation and organizational hierarchy, making subject matter easily assessed. Professional design conveys authority and trust, making viewers more likely to visit and backlink to your site. Finally, high-quality content will bring links and invite viewers to spend more time on your site.

What You Can Do

  • Do some research and define your audience: What are potential users of your site most likely to search for?
  • Make your keywords prominent by putting them in important locations such as the title of your page.
  • Choose specific keywords relevant to your niche. Generic keywords invite competition SERPs. Choose one or two unique phrases to target per page on a new site.
  • Make your web address brief and descriptive. You can include keywords here, as well.
  • Write quality copy for your site. Search engines use lexical analysis to judge quality and rank pages, so hire a copy editor if you need to.
  • Include a site map that offers links to your site’s internal pages. 
  • Make sure your designer can generate clean code. Invalid HTML and CSS, broken links and large file sizes negatively affect a site’s ability to be crawled.
  • Avoid designing with nested tables, which break up text and make it difficult for a page to be crawled. Sites created in the past year and later should include smart usage of CSS. Older sites relying on tables should consider upgrading their page layouts.

The Bottom Line

Even professional search engine optimizers are unaware of the exact procedure search engines use to rank pages. If all the secrets were released, the process could be cheated, rendering search engines less effective at returning relevant information. The bottom line is that high rankings come from high-quality pages with professional design and great content. Follow that mantra and you have taken the biggest step toward obtaining a high ranking.