SEO techniques that are based on Google's SEO Articles ( part 9 )

Creating an effective robot.txt
Robots.txt is a file that ordered the search engines CAN NOT access a specific section or in a web site. For example on a web site http://tutorialgratis.net, Robots.txt is located at: http://tutorialgratis.net/robots.txt.

    Sitemap: http://ghamebangget.blogspot.com/sitemap.xml
    User-agent: IRLbot
    Crawl-delay: 3600
    User-agent: *
    Disallow: / next /
    User-agent: *
    Disallow:
If the code is entered on the robots.txt file to Disallow a specific folder, the contents of these folders will not be indexed search engines, but can appear in search results, if there is a link to that page on other pages (including referrer logs). For more details, see the tutorial robots.txt.
So robots.txt to prevent search engines from indexing certain pages but not block the visitors because visitors can still open it if you know its URL.
Search Suggestions:

    * For sensitive pages that do not just use robots.txt security protection such as htaccess.
Common mistakes in SEO that should be avoided:

    * Allow search result search engine indexed.
    * Allowing similar pages indexed search engines.
    * Allows URLs from search engines indexed a proxy server.