Comments on: Validate your robots.txt - Googlebot becomes smarter http://sebastians-pamphlets.com/validate-your-robots-txt-or-google-might-deindex-your-site/ If you've read my articles somewhere on the Internet, expect something different here. Fri, 25 Jul 2008 11:19:35 +0000 http://wordpress.org/?v=2.2.3 By: My plea to Google - Please sanitize your REP revamps http://sebastians-pamphlets.com/validate-your-robots-txt-or-google-might-deindex-your-site/#comment-916 My plea to Google - Please sanitize your REP revamps Fri, 04 Jan 2008 00:03:12 +0000 http://sebastians-pamphlets.com/validate-your-robots-txt-or-google-might-deindex-your-site/#comment-916 [...] supports a Noindex: directive in robots.txt. It seems Google’s Noindex: blocks crawling like Disallow:, but additionally prevents URLs blocked with [...]

[…] supports a Noindex: directive in robots.txt. It seems Google’s Noindex: blocks crawling like Disallow:, but additionally prevents URLs blocked with […]

]]> By: Utah SEO Pro http://sebastians-pamphlets.com/validate-your-robots-txt-or-google-might-deindex-your-site/#comment-777 Utah SEO Pro Sun, 02 Dec 2007 20:37:12 +0000 http://sebastians-pamphlets.com/validate-your-robots-txt-or-google-might-deindex-your-site/#comment-777 Oh my god, block level robots.txt commands is ingenious! That'd be great. I always thought there should be an inline directive for that. Oh my god, block level robots.txt commands is ingenious! That’d be great. I always thought there should be an inline directive for that.

]]> By: Google to change the Robots Exclusion Protocol again http://sebastians-pamphlets.com/validate-your-robots-txt-or-google-might-deindex-your-site/#comment-775 Google to change the Robots Exclusion Protocol again Sat, 01 Dec 2007 09:42:54 +0000 http://sebastians-pamphlets.com/validate-your-robots-txt-or-google-might-deindex-your-site/#comment-775 [...] Google is working on new robots.txt syntax, and I am, politely put, not amused. Here is why I fear that Google is going to totally mess up the [...] […] Google is working on new robots.txt syntax, and I am, politely put, not amused. Here is why I fear that Google is going to totally mess up the […]

]]> By: Sebastian http://sebastians-pamphlets.com/validate-your-robots-txt-or-google-might-deindex-your-site/#comment-764 Sebastian Wed, 28 Nov 2007 18:55:40 +0000 http://sebastians-pamphlets.com/validate-your-robots-txt-or-google-might-deindex-your-site/#comment-764 Looking at the <a href="http://sebastians-pamphlets.com/validate-your-robots-txt-or-google-might-deindex-your-site/#robots-txt-test-results-2007-11-28">preliminary test results</a>, I was not amused. Looking at the preliminary test results, I was not amused.

]]> By: david deangelo http://sebastians-pamphlets.com/validate-your-robots-txt-or-google-might-deindex-your-site/#comment-734 david deangelo Sat, 24 Nov 2007 07:57:41 +0000 http://sebastians-pamphlets.com/validate-your-robots-txt-or-google-might-deindex-your-site/#comment-734 Good article. My main use of robot.txt is to disallow certain bots. Good article. My main use of robot.txt is to disallow certain bots.

]]> By: 5 ways to de-index your site • Tim Nash UK SEO Blog http://sebastians-pamphlets.com/validate-your-robots-txt-or-google-might-deindex-your-site/#comment-731 5 ways to de-index your site • Tim Nash UK SEO Blog Fri, 23 Nov 2007 08:36:56 +0000 http://sebastians-pamphlets.com/validate-your-robots-txt-or-google-might-deindex-your-site/#comment-731 [...] in 94 now in the nauties we can simply screw up our site with the line Noindex: / How cool is that! Sebastian as always has come up with some more creative examples of robot.txt directives which may or may not [...] […] in 94 now in the nauties we can simply screw up our site with the line Noindex: / How cool is that! Sebastian as always has come up with some more creative examples of robot.txt directives which may or may not […]

]]> By: Search Engine Land: News About Search Engines & Search Marketing http://sebastians-pamphlets.com/validate-your-robots-txt-or-google-might-deindex-your-site/#comment-727 Search Engine Land: News About Search Engines & Search Marketing Wed, 21 Nov 2007 17:41:00 +0000 http://sebastians-pamphlets.com/validate-your-robots-txt-or-google-might-deindex-your-site/#comment-727 <strong>SearchCap: The Day In Search, November 21, 2007...</strong> Below is what happened in search today, as reported on Search Engine Land and from other places across the web....... SearchCap: The Day In Search, November 21, 2007…

Below is what happened in search today, as reported on Search Engine Land and from other places across the web…….

]]> By: kelvin newman http://sebastians-pamphlets.com/validate-your-robots-txt-or-google-might-deindex-your-site/#comment-724 kelvin newman Wed, 21 Nov 2007 13:30:27 +0000 http://sebastians-pamphlets.com/validate-your-robots-txt-or-google-might-deindex-your-site/#comment-724 I really ought to get to grips with Robots.txt stuff, I find it intriguing even though it's not an area of expertise I really ought to get to grips with Robots.txt stuff, I find it intriguing even though it’s not an area of expertise

]]> By: Time to Check Your Robots.txt | The Search Engine Blog http://sebastians-pamphlets.com/validate-your-robots-txt-or-google-might-deindex-your-site/#comment-722 Time to Check Your Robots.txt | The Search Engine Blog Wed, 21 Nov 2007 11:43:33 +0000 http://sebastians-pamphlets.com/validate-your-robots-txt-or-google-might-deindex-your-site/#comment-722 [...] Validate your robots.txt - Googlebot becomes smarter [...] […] Validate your robots.txt - Googlebot becomes smarter […]

]]> By: Steaprok http://sebastians-pamphlets.com/validate-your-robots-txt-or-google-might-deindex-your-site/#comment-721 Steaprok Wed, 21 Nov 2007 10:52:14 +0000 http://sebastians-pamphlets.com/validate-your-robots-txt-or-google-might-deindex-your-site/#comment-721 Great Post Sebastian! I have come across this post several times already, on Sphinn, on SER, and others. Awesome insight.. and with confirmation from Goog too, nice! and thanks for the comment on my Blog. Great Post Sebastian! I have come across this post several times already, on Sphinn, on SER, and others. Awesome insight.. and with confirmation from Goog too, nice!

and thanks for the comment on my Blog.

]]>