Website Architecture
The structural layout of your website makes a big difference for the user. Poorly constructed websites are usually the result of a small site gradually evolving into a more complex one, with additional departments and pages duct taped onto an old infrastructure. These types of sites are easy to get lost in; stuck is a dark corner with no links leading you back. We’ve all seen them.

Search engines don’t like to send their users to such sites, as it makes for a bad user experience. Crawling robots can actually determine the layout and organization of a site, based on it’s URL structure, site map and file hierarchy. Meaning, logical and intuitive architecture is quite important to earn good search results.

File layout
Like traditional architecture, website construction involves the balance of technical, aesthetic and functional criteria. To achieve this balance, a level system is used, just as the file system on your computer, from general to more specific topics. The homepage acts as the first level, broken down into natural second levels, usually categories. Each category contains sub-categories, making a hierarchy that flows smoothly as the site expands.

File naming
As the site hierarchy is created, it is important that the developer logically name each file, as these will translate into the URL for each page. Names that include descriptive and relevant keywords will help search engines know exactly what each page is about, so they can be correctly indexed and show up for relevant queries. Also, as edits are made down the road, it will make life easier by using a simple naming convention, allowing developers to quickly find the needed files to be edited. Two conventions should be followed when naming your site files:

Relevant Keywords: Including phrases in the file directory will help with search engine results. However, you will want to avoid stuffing too many keywords into the name.

Good example:
www.gravitateonline.com/services/local-seo/

Poor example:
www.gravitateonline.com/category_3/seo%_1&id=32

Keyword stuffed example:
www.gravitateonline.com/online-marketing-services-for-    utah/local-seo-for-lead-generation/

Dash Separated: A debate has been going on within the SEO world for quite some time about the use of underscores (i.e. plumbing_services ) versus dashes (i.e. plumbing-services) in naming files. Matt Cutts from Google has addressed this principle:

“I often get asked whether I’d recommend dashes or underscores for words in urls. For urls in Google, I would recommend using dashes….With underscores, Google’s programmer roots are showing. Lots of computer programming languages have stuff like _MAXINT, which may be different than MAXINT. So if you have a url like word1_word2, Google will only return that page if the user searches for word1_word2 (which almost never happens). If you have a url like word1-word2, that page can be returned for the searches word1, word2, and even “word1 word2″.

At Gravitate Online, we follow Google’s suggestion and use dashes in naming all pages and files on your website, making it easier for Google to read your keywords and rank you higher for them.

Robot.txt File
The Robot.txt is a text file, not HTML, that is place in your site’s root directory that tells search engines which pages to crawl and not to crawl. By default, every page is crawl-able, but in some cases, there are pages you may not want to be found in search engines. Basically, it’s a “Do Not Enter” sign for certain pages.  You may not want Google to index your image pages, or employee only pages. Also, indexing non-critical pages may waste site bandwidth.

The file itself is fairly small and includes straight forward instructions to search engine spiders on which pages are to be crawled and indexed, for example the instructions may include:

Allow all robots to visit all files on site:

User-agent: *
Disallow:

Dissallow allow all robots from the certain pages on the site:

User-agent: *
Disallow: /cgi-bin/
Disallow: /tmp/
Disallow: /private_pages/