Robots.txt isn’t something that’s often talked about in articles about SEO, but it is an important consideration. Most sites will have some files that they don’t want indexed for various reasons. The function of your robots.txt files, however, isn’t a simple matter of ‘allow’ and ‘disallow’. Your site’s code plays an important role in the Read Blog…
Avoid messing up your robots.txt
December 14th, 2009 by Jon









