As I’m sure you know, a robots.txt file is a pretty handy way for webmasters to tell bots that they don’t want them crawling around specific pages/sections within your site. It’s not essential to have a robots.txt file but if you do, make sure it is reachable! I’ve found that if your robots.txt isn’t reachable Read Blog…
Ensure Your robots.txt File Is Reachable
June 25th, 2011 by Peter
A Quick Guide To The Robots Meta Tag, “noindex,follow” Attribute
December 14th, 2010 by Peter
As-well as the robots.txt which allows you to control the way robots behave within your website, there’s the robots meta tag. Which when used right, can be much more flexible than the robots.txt. The noindex,follow attribute is supported by all major search engines as far as I’m aware (Google, Yahoo! and Live). There are many Read Blog…
SEO: Don’t Duplicate, Innovate
February 2nd, 2010 by Jon
Google considers duplicate content to be “substantive” blocks of text within a single domain or across multiple domains that are either completely identical or are “appreciably similar”. Duplicate content is generally a bad thing as some use it as an attempt to influence search engines into giving stronger rankings to pages. This is bad for Read Blog…
How robots.txt Can Affect Search Engine Optimisation Strategy
December 21st, 2009 by Nick
The search engine optimisation of a site involves a lot of different things. Some of these are very technical aspects, while others of them are substantially more creative. The site’s architecture, code and content all have a very important role to play in where the search engines place its link in the search engine results Read Blog…
Avoid messing up your robots.txt
December 14th, 2009 by Jon
Robots.txt isn’t something that’s often talked about in articles about SEO, but it is an important consideration. Most sites will have some files that they don’t want indexed for various reasons. The function of your robots.txt files, however, isn’t a simple matter of ‘allow’ and ‘disallow’. Your site’s code plays an important role in the Read Blog…









