Comments on: Google to change the Robots Exclusion Protocol again

By: My plea to Google - Please sanitize your REP revamps

My plea to Google - Please sanitize your REP revamps — Fri, 04 Jan 2008 00:02:56 +0000

[…] Google experiments with new robots.txt directives, that is REP tags like “noindex” adapted for robots.txt. That’s a welcomed and […]

By: Utah SEO

Utah SEO — Tue, 25 Dec 2007 00:30:02 +0000

yeah, block-level directives would kick ass

By: Search Engine Land: News About Search Engines & Search Marketing

Search Engine Land: News About Search Engines & Search Marketing — Mon, 03 Dec 2007 18:14:56 +0000

SearchCap: The Day In Search, December 3, 2007…

Below is what happened in search today, as reported on Search Engine Land and from other places across the web…….

By: Matt

Matt — Mon, 03 Dec 2007 14:09:25 +0000

I guess people reacted the same way when the no follow was added, but now some people think it’s a great thing…I’m not so sure this or the no follow are actual benefits to the internet…seems Google is impeding the growth of the net with all these stipulations

By: david deangelo

david deangelo — Mon, 03 Dec 2007 08:43:41 +0000

Thanks Sebastian. I think i will not use noindex: in the robots.txt until its become official. I only use the disallow: on redirected URLs e.g. affiliate links.

By: Sebastian

Sebastian — Mon, 03 Dec 2007 08:34:32 +0000

“Disallow:” forbids crawling, but allows URLs to gather PageRank, anchor text and whatnot. That’s old news. The new “Noindex:” directive in robots.txt would block PageRank flow and indexing under anchor text or titles/descriptions of 3rd party links, but it’s experimental. As long as Google doesn’t announce it there’s no official support, and the behavior described in my post can change without notice. Currently there’s no sure-fire way to prevent Disallow’ed URLs from getting indexed respectively from accumulating link juice of any kind.

By: david deangelo

david deangelo — Mon, 03 Dec 2007 08:07:03 +0000

I have always thought disallow: would not allow a page to be crawled and hence would not allow page rank to accummulate. Does that mean I should use noindex: too for the pagerank to to go to those pages from links on my other pages?

By: Michael VanDeMar

Michael VanDeMar — Sat, 01 Dec 2007 00:01:44 +0000

Google must not redefine syntax and semantics that were in widespread use before Google even existed. I admit they’ve the power to fuck up the REP, but they also have “do no evil”.

Can we really still claim that we believe that “Don’t Be Evil” remains a part of their creed? Just because the phrase “absolute power corrupts absolutely” is a cliche does not mean that it doesn’t still hold true in many cases.

Currently the only thing that Google seems to care about, the only thing that is likely to curb them from doing things with a negative impact, is image, and there only if it is something that will cause a stir with the mainstream. With something as technical as this I doubt that enough of a buzz will occur for them to even consider not screwing with it.