How The Googlebot Works In Relation To Your SEO Campaign
November 5th, 2008 by Nick
Search engine optimization is about making your website as ’search engine friendly’ as possible, thus achieving a high page rank when people do searches. There are so many pages on the web that you cannot rely on your website being found with a hit and miss strategy. Even if it is indexed in a search return, there may be thousands of returns – and people are not going to wade through them all. The contest is to get into the top ten, or even the top twenty.
Googlebot
Googlebot is the web crawler used by Google. It crawls the web searching for pages, and then passes them on to the Indexer, which sorts every word and then stores those pages for future searches. It starts with a list of URLs, then investigates hyperlinks on the page, and adds them to its list of URLs to search.
Two of the main problems for search engines are the huge number of pages on the web, and the fact that fresh information is being produced all the time. For that reason Googlebot has two versions, the Deepbot for deep searches, and the Freshbot, which only looks for fresh content.
Googlebot discovers pages by investigating the links from every page that it searches. For it to find a web page, it needs to be pointed there from a link on an existing page that it visits.
Searchable content
Because of the way the crawler works, it is important to use a number of links to other sites, and to other pages on your site, to point the crawlers to your page. It is particularly useful to have links to sites that achieve a high page ranking. Reciprocal site links are understood to be less effective than One Way Links; therefore interaction with other sites is advantageous but only if they are of excellent quality to the user.
Meta tags and keywords are important too, so that the indexers can clearly distinguish the pertinent content on your page, in order to sort it for search queries. However, pointless keyword usage to maximise keyword density can have a negative effect, as it looks like spam and your site can be ‘blacklisted’ by the crawler. More important is to have good quality content, with only necessary keywords.
Fresh content on your site is crucial. Once your site has been discovered and is being listed for visits, regular Fresh Content will ensure more regular visits. The more popular the site, and the higher the page ranking, the more regularly the Freshbot will check it.
SPAM and what not to do
If your site is identified as spamming, it will be excluded from further crawls. Keyword density for the sake of it could easily result in your site’s exclusion.
One may have to protect one’s site from other people’s spamming, too. For example, there’s a known practice of people leaving comments on your site purely for the purpose of creating a link back to theirs. If one suspects this practice, one can follow a “no-follow” strategy, whereby you specifically tell the crawler not to visit those links.
You can follow any responses to this entry through the
RSS feed.
Link to us
If you want to link to this blog, copy and paste the following HTML code to your website.
Hi Nick,
I’m running a series of IR tutorials on my blog at the moment, I kicked off with “The search engine index” :
http://scienceforseo.blogspot.com/2008/11/search-engine-index-tutorial.html
Would you like to stick your article in a PPT presentation with some diagrams and stick it in Slideshare? Then I’ll add it to the series and put it on the blog.
Twitter: Missmcj
In the respect of search engine optimization Google bot plays important role in serving the SERP of the site. Google Bot visits the sites if it finds appropriate according to its algorithms gives them ranking.
For Google search engine Google bot is very important agent for visiting the over internet for crawling and reading, it collects those sites which are deputed relevant and appropriate keywords sites theme.