Google Changes SERP Description For Robots.txt Blocked Pages
If you block a page within the robots.txt, Google still often indexes the page however it doesn’t crawl the page. This means that the meta description and meta title isn’t displayed within the SERPs.
The title within the SERPs which is displayed instead, is usually the full URL, like this:
If the blocked page has been linked to (either internally or externally), the anchor text of one of the links directing to the page is often used as the title instead, like this:
Up until today though, the meta description has always been blank for blocked pages. Earlier this morning however, I noticed that this had been changed. Now, for blocked pages indexed within Google, in place of the meta description, there’s now a reason why there’s no description. The description/reason now reads "A description for this result is not available because of this site's robots.txt – learn more.". This is how a blocked listing now looks:
Pretty interesting! Let us know your thoughts below.
