Unavailable_After is totally and utterly useless

I’ve a lot of respect for Dan Crow, but I’m struggling with my understanding, or possible support, of the unavailable_after tag. I don’t want to put my reputation for bashing such initiatives from search engines at risk, so sit back and grab your popcorn, here comes the roasting:

As a Webmaster, I did not find a single scenario where I could or even would use it. That’s because I’m a greedy traffic whore. A bazillion other Webmasters are greedy too. So how the heck is Google going to sell the newish tag to the greedy masses?

Ok, from a search engine’s perspective unavailable_after makes sound sense. Outdated pages bind resources, annoy searchers, and in a row of useless crap the next bad thing after an outdated page is intentional Webspam.

So convincing the great unwashed to put that thingy on their pages inviting friends and family to granny’s birthday party on 25-Aug-2007 15:00:00 EST would improve search quality. Not that family blog owners care about new meta tags, RFC 850-ish date formats, or search engine algos rarely understanding that the announced party is history on Aug/26/2007. Besides there may be painful aftermaths worth submitting a desperate call for aspirins the day after in the comments, what would be news of the day after expiration. Kinda dilemma, isn’t it?

Seriously, unless CMS vendors support the new tag, tiny sites and clique blogs aren’t Google’s target audience. This initiative addresses large sites which are responsible for a huge amount of outdated contents in Google’s search index.

So what is the large site Webmaster’s advantage of using the unavailable_after tag? A loss of search engine traffic. A loss of link juice gained by the expired page. And so on. Losses of any kind are not that helpful when it comes to an overdue raise nor in salary negotiations. Hence the Webmaster asks for the sack when s/he implements Google’s traffic terminator.

Who cares about Google’s search quality problems when it leads to traffic losses? Nobody. Caring Webmasters do the right thing anyway. And they don’t need no more useless meta tags like unavailable_after. “We don’t need no stinking metas” from “Another Brick in the Wall Part Web 2.0″ expresses my thoughts perfectly.

So what separates the caring Webmaster from the ‘ruthless traffic junky’ who Google wants to implement the unavailable_after tag? The traffic junkie lets his stuff expire without telling Google about it’s state, is happy that frustrated searchers click the URL from the SERPs even years after the event, and enjoys the earnings from tons of ads placed above the content minutes after the party was over. Dear Google, you can’t convince this guy.

[It seems this is a post about repetitive “so whats”. And I came to the point before the 4th paragraph … wow, that’s new … and I’ve put a message in the title which is not even meant as link bait. Keep on reading.]

So what does the caring Webmaster do without the newish unavailable_after tag? Business as usual. Examples:

Say I run a news site where the free contents go to the subscription area after a while. I’d closely watch which search terms generate traffic, write a search engine optimized summary containing those keywords, put that on the sales pitch, and move the original article to the archives accessible to subscribers only. It’s not my fault that the engines think they point to the original article after the move. When they recrawl and reindex the page my traffic will increase because my summary fits their needs more perfectly.

Say I run an auction site. Unfortunately particular auctions expire, but I’m sure that the offered products will return to my site. Hence I don’t close the page, but I search my database for similar offerings and promote them under a H3 heading like “[product] (stuffed keywords) is hot” /H3 P buy [product] here: /P followed by a list of identical products for sale or similar auctions.

Say I run a poll expiring in two weeks. With Google’s newish near real time indexing that’s enough time to collect keywords from my stats, so the textual summary under the poll’s results will attract the engines as well as visitors when the poll is closed. Also, many visitors will follow the links to related respectively new polls.

From Google’s POV there’s nothing wrong with my examples, because the visitor gets what s/he was searching for, and I didn’t cheat. Now tell me, why should I give up these valuable sources of nicely targeted search engine traffic just to make Google happy? Rather I’d make my employer happy. Dear Google, you didn’t convince me.

Update: Tanner Christensen posted a remarkable comment at Sphinn:

I’m sure there is some really great potential for the tag. It’s just none of us have a need for it right now.

Take, for example, when you buy your car without a cup holder. You didn’t think you would use it. But then, one day, you find yourself driving home with three cups of fruit punch and no cup holders. Doh!

I say we wait it out for a while before we really jump on any conclusions about the tag.

John Andrews was the first to report an evil use of unavailable_after.

Also, Dan Crow from Google announced a pretty neat thing in the same post: With the X-Robots-Tag you can now apply crawler directives valid in robots meta tags to non-HTML documents like PDF files or images.

Share/bookmark this: del.icio.usGooglema.gnoliaMixxNetscaperedditSphinnSquidooStumbleUponYahoo MyWeb
Subscribe to      Entries Entries      Comments Comments      All Comments All Comments

6 Comments to "Unavailable_After is totally and utterly useless"

  1. Matt Stoddart on 31 July, 2007  #link


    And this is brilliant, “We don’t need no stinking metas” from “Another Brick in the Wall Part Web 2.0″

  2. neuro on 31 July, 2007  #link

    yeh But….

    I bet this is so they can say to newspaper publishers (or courts) like the ones who took them to court.

    “here is the mechanisiam for you protect your paid for archives so you can stuff your rinkiny dink protectionist court case up your ass”

  3. Sebastian on 31 July, 2007  #link

    Thanks Matt :)

    Neuro, I agree that possibly newspaper publishers *are* Google’s ‘targeted audience’ WRT to unavailable_after. However, even if it is a CYA it deserves a comment when it is announced as a call for action, inviting you and me and everyone. An announcement like “dear newspapers, if you don’t want us caching your stuff and sending you free traffic after the initial coverage, just implement unavailable_after, a REP tag we’ve created for you guys.” would have prevented me from writing this piece. I admit that when the background is legal pressure, Google does not deserve bashing. I think my post doesn’t bash Google. If so, I apologize.

  4. […] us robots meta tags for non-HTML resources like PDF documents, images or video clips, and with Unavailable_after Google made a few clueless news sites happy. With the rel-nofollow microformat on the other hand, […]

  5. Stuart Clark on 17 August, 2009  #link

    Actually it’s pretty useful in email marketing.

    I work for a company where our clients import HTML into the system and send it out in bulk as emails (don’t worry, I’m not talking about spam, it’s all legit).

    All of the emails are available as webpages until clients delete them (which they never do). With Unavailable_After, our clients can set an expiry on their newsletters so tons of several year old emails don’t show up on the internet. So it does have it’s uses!

  6. LazNiko on 29 September, 2011  #link

    Well, can’t think of a situation to use something doesn’t mean that something is useless.

    We have a website where users post classified ads on it. Each ad has an “expiry date”. After the date, we want the ad become abandoned and non-search-able by others. Also, after the expiry date, some users may post a new ad with 90% content exactly same as the old ad, and the expiry date is the only difference. They use this approach to “extend” the lifetime of an ad.

    Yes, removing the page from search engine may lose traffic, but if not to do so,the website will end up with a lot of duplicated and outdated pages in the search engine’s view.

    So what’s your advice to deal with this if there is no “unavailable_after”?

Leave a reply

[If you don't do the math, or the answer is wrong, you'd better have saved your comment before hitting submit. Here is why.]

Be nice and feel free to link out when a link adds value to your comment. More in my comment policy.