Comments on: Why storing URLs with truncated trailing slashes is an utterly idiocy http://sebastians-pamphlets.com/thou-must-not-steal-the-trailing-slash-from-my-urls/ If you've read my articles somewhere on the Internet, expect something different here. Fri, 25 Jul 2008 11:18:56 +0000 http://wordpress.org/?v=2.2.3 By: Kenpachi http://sebastians-pamphlets.com/thou-must-not-steal-the-trailing-slash-from-my-urls/#comment-1304 Kenpachi Wed, 26 Mar 2008 19:29:21 +0000 http://sebastians-pamphlets.com/thou-must-not-steal-the-trailing-slash-from-my-urls/#comment-1304 Awesome article! I was recently having issues with my site and Google Sitemaps, Googlebot kept encoutering 302 redirect errors. After some reearch I found that google expects your urls in the sitemap to end with a trailing slash. End story was I had to recode a few parts, and I had to add a slash to the end of every url, since I make heavy use of URL re-writing. I was about to look for a way to make my site "look cool" by having an option for a non-trailing urls, but your article straightened me out :)

Awesome article! I was recently having issues with my site and Google Sitemaps, Googlebot kept encoutering 302 redirect errors. After some reearch I found that google expects your urls in the sitemap to end with a trailing slash.

End story was I had to recode a few parts, and I had to add a slash to the end of every url, since I make heavy use of URL re-writing. I was about to look for a way to make my site “look cool” by having an option for a non-trailing urls, but your article straightened me out :)

]]> By: Matt N http://sebastians-pamphlets.com/thou-must-not-steal-the-trailing-slash-from-my-urls/#comment-1298 Matt N Thu, 20 Mar 2008 16:35:10 +0000 http://sebastians-pamphlets.com/thou-must-not-steal-the-trailing-slash-from-my-urls/#comment-1298 Yeah there is a lot of misunderstanding about URI's when it comes to this department. This W3C document helped me a lot: http://www.w3.org/TR/chips/#uri The thing that stands out to me most (and what most people misunderstand about URI's because of Apache's default configuration) is that a URI IS NOT a filesystem. Apache uses that as a convenient default because it's a 'quick and dirty' way to map URI's to resources on the filesystem. Apache provides many methods for making this mapping other than the filesystem. Modules such as mod_alias, mod_rewrite and mod_negotiation (as well as many others) are much more advanced methods of configuring your server for mapping URI's to resources on the filesystem. The 'redirects that the server does automatically' involving trailing slashes with URI's is actually due to an Apache module called mod_speling. This module is activated in Apache's default install and is one of the causes of this confusion! Yeah there is a lot of misunderstanding about URI’s when it comes to this department. This W3C document helped me a lot:

http://www.w3.org/TR/chips/#uri

The thing that stands out to me most (and what most people misunderstand about URI’s because of Apache’s default configuration) is that a URI IS NOT a filesystem. Apache uses that as a convenient default because it’s a ‘quick and dirty’ way to map URI’s to resources on the filesystem. Apache provides many methods for making this mapping other than the filesystem. Modules such as mod_alias, mod_rewrite and mod_negotiation (as well as many others) are much more advanced methods of configuring your server for mapping URI’s to resources on the filesystem.

The ‘redirects that the server does automatically’ involving trailing slashes with URI’s is actually due to an Apache module called mod_speling. This module is activated in Apache’s default install and is one of the causes of this confusion!

]]> By: Sebastian http://sebastians-pamphlets.com/thou-must-not-steal-the-trailing-slash-from-my-urls/#comment-1282 Sebastian Thu, 06 Mar 2008 08:23:16 +0000 http://sebastians-pamphlets.com/thou-must-not-steal-the-trailing-slash-from-my-urls/#comment-1282 Tristan, seems you're right, I don't consider developers plain users. As for users navigating the Web in the browser address bar, well, search engines do a great job leading type-in queries to the best matching result, or a SERP. That's a great and useful layer between humans and URIs, and it should get improved further, beyond autocomplete, ajax'ed suggestions and such neat stuff. I strongly believe that users shouldn't deal with URIs at all, I mean that's why we've links with meaningful anchor text. But as long as I can't click a link on my telly or on a placard, type-in traffic exists, albeit it lands at quite exotic and unrelated destinations in many cases. The challenge for developers is to keep users away from URIs wherever possible. Tristan, seems you’re right, I don’t consider developers plain users. As for users navigating the Web in the browser address bar, well, search engines do a great job leading type-in queries to the best matching result, or a SERP. That’s a great and useful layer between humans and URIs, and it should get improved further, beyond autocomplete, ajax’ed suggestions and such neat stuff.

I strongly believe that users shouldn’t deal with URIs at all, I mean that’s why we’ve links with meaningful anchor text. But as long as I can’t click a link on my telly or on a placard, type-in traffic exists, albeit it lands at quite exotic and unrelated destinations in many cases. The challenge for developers is to keep users away from URIs wherever possible.

]]> By: Tristan http://sebastians-pamphlets.com/thou-must-not-steal-the-trailing-slash-from-my-urls/#comment-1281 Tristan Thu, 06 Mar 2008 07:53:57 +0000 http://sebastians-pamphlets.com/thou-must-not-steal-the-trailing-slash-from-my-urls/#comment-1281 I think our problem here is that we're talking about different things. I'm talking about users typing in URIs and you're talking about links and references created by developers or applications. From a user's point of view when typing the URI the slash is meaningless (since the content is the same in 99% of cases) and that will never change, nor should it ever, since it's in a server's best interest to allow both. From a machine's perspective, it's best to have a single canonical URI. Thus, redirects for the users and slashes for the machines. Best of both worlds. So, we were on completely different tracks... sorry for the confusion, I thought it was clear I was talking about users. But if you go back to what you wrote here.... "Tristan, when a user has to see an URI (on the Interweb) and can click it, it should be the canonical URI. If a user has to read or hear an URI (elsewhere) and is supposed to remember it, it should be as short as possible. The machines populating todays Web aren’t smart enough to deal with “offline inputs”." --- That I agree with completely and let's leave it at that. I was just interested in discussing deeper issues of human-centric issues around URI and their "correct" behavior, but apparently you aren't. I think our problem here is that we’re talking about different things. I’m talking about users typing in URIs and you’re talking about links and references created by developers or applications. From a user’s point of view when typing the URI the slash is meaningless (since the content is the same in 99% of cases) and that will never change, nor should it ever, since it’s in a server’s best interest to allow both. From a machine’s perspective, it’s best to have a single canonical URI. Thus, redirects for the users and slashes for the machines. Best of both worlds.

So, we were on completely different tracks… sorry for the confusion, I thought it was clear I was talking about users.

But if you go back to what you wrote here…. “Tristan, when a user has to see an URI (on the Interweb) and can click it, it should be the canonical URI. If a user has to read or hear an URI (elsewhere) and is supposed to remember it, it should be as short as possible. The machines populating todays Web aren’t smart enough to deal with “offline inputs”.”
— That I agree with completely and let’s leave it at that. I was just interested in discussing deeper issues of human-centric issues around URI and their “correct” behavior, but apparently you aren’t.

]]> By: Sebastian http://sebastians-pamphlets.com/thou-must-not-steal-the-trailing-slash-from-my-urls/#comment-1278 Sebastian Thu, 06 Mar 2008 06:29:01 +0000 http://sebastians-pamphlets.com/thou-must-not-steal-the-trailing-slash-from-my-urls/#comment-1278 Tristan, the phone number example totally makes sense, and that's the point you don't get. The trailing slash is part of the address. When you remove it in some cases you get different content, and in some cases you run in a redirect. A manipulated URI is like a different phone number. When you add a character to an URI, as you do with your additional digit, you will always run in a 404-Not found error on the Web. Adding stuff to URIs is as bad as truncating them in hyperlinks. With phone numbers of call centers your additional digit will be taken as input after the connection was established, so you can end up in a support queue instead of chatting with a sales rep. So the lesson is, don't play with addresses of any kind. Frankly, neither stealing the trailing slash from URIs nor adding or removing digits from phone numbers is "natural human behavior". It's a plain stupid error that kids learn to avoid in elementary school, if they didn't figure it out already by applying the trial-and-error method. In other words, in some cases folks just have to play by the rules, because they land nowhere otherwise - online as well as offline. I don't see a way to improve the current error handling in both cases (invalid URI / phone#), and there's absolutely no reason to encourage users to use incorrect respectively manipulated identifiers. BTW a directory URI request without trailing slash runs in a 404 internally before a well configured Web server recognizes that "the user is simply broken" and replies with a redirect header. "Machine-centric viewpoints" can save the human vistors time, and hassles. ;) Tristan, the phone number example totally makes sense, and that’s the point you don’t get. The trailing slash is part of the address. When you remove it in some cases you get different content, and in some cases you run in a redirect. A manipulated URI is like a different phone number. When you add a character to an URI, as you do with your additional digit, you will always run in a 404-Not found error on the Web. Adding stuff to URIs is as bad as truncating them in hyperlinks. With phone numbers of call centers your additional digit will be taken as input after the connection was established, so you can end up in a support queue instead of chatting with a sales rep. So the lesson is, don’t play with addresses of any kind. Frankly, neither stealing the trailing slash from URIs nor adding or removing digits from phone numbers is “natural human behavior”. It’s a plain stupid error that kids learn to avoid in elementary school, if they didn’t figure it out already by applying the trial-and-error method. In other words, in some cases folks just have to play by the rules, because they land nowhere otherwise - online as well as offline. I don’t see a way to improve the current error handling in both cases (invalid URI / phone#), and there’s absolutely no reason to encourage users to use incorrect respectively manipulated identifiers. BTW a directory URI request without trailing slash runs in a 404 internally before a well configured Web server recognizes that “the user is simply broken” and replies with a redirect header. “Machine-centric viewpoints” can save the human vistors time, and hassles. ;)

]]> By: Tristan http://sebastians-pamphlets.com/thou-must-not-steal-the-trailing-slash-from-my-urls/#comment-1272 Tristan Thu, 06 Mar 2008 03:16:04 +0000 http://sebastians-pamphlets.com/thou-must-not-steal-the-trailing-slash-from-my-urls/#comment-1272 That's exactly right in today's Internet, in the world we live in, in reality. That's what you're talking about. All I'm trying to do is discuss how we might improve the state of the system to better support both humans and machines, which is what the system designed for machines only is doing right now with hacks (redirects) to support natural human error. Your example with phone numbers doesn't make sense anyway. Even in today's world, the real analogy is as if there was an extra "0" at the end of each phone number (so, 555-01000, 555-01010, 555-01090), and it didn't really matter if you included it or not (555-0100, 555-0101, 555-0109 still worked), it would still get you to the same place (because that's how URIs *functionally* work today on the real internet). Would you still dial the extra 0 at the end just out of convention? No, of course not. Okay okay, I'm not arguing anymore. I understand and agree with you, I was just put off by your machine-centric viewpoint. I suppose it's the individual site/server's responsibility to support user URI errors such as missing slashes, and then to use correct URIs for all external links and resources. I just want to make sure you understand that, for users' sake, a resource without a slash had darn well better mean the same thing as with the slash. If a domain without a slash gave a 404, then for all usability purposes it is simply broken. So, that too is the responsibility of the site to accommodate its users, which is equal to or greater than its responsibility to other sites on the internet. That’s exactly right in today’s Internet, in the world we live in, in reality. That’s what you’re talking about.

All I’m trying to do is discuss how we might improve the state of the system to better support both humans and machines, which is what the system designed for machines only is doing right now with hacks (redirects) to support natural human error.

Your example with phone numbers doesn’t make sense anyway. Even in today’s world, the real analogy is as if there was an extra “0″ at the end of each phone number (so, 555-01000, 555-01010, 555-01090), and it didn’t really matter if you included it or not (555-0100, 555-0101, 555-0109 still worked), it would still get you to the same place (because that’s how URIs *functionally* work today on the real internet). Would you still dial the extra 0 at the end just out of convention? No, of course not.

Okay okay, I’m not arguing anymore. I understand and agree with you, I was just put off by your machine-centric viewpoint. I suppose it’s the individual site/server’s responsibility to support user URI errors such as missing slashes, and then to use correct URIs for all external links and resources.

I just want to make sure you understand that, for users’ sake, a resource without a slash had darn well better mean the same thing as with the slash. If a domain without a slash gave a 404, then for all usability purposes it is simply broken. So, that too is the responsibility of the site to accommodate its users, which is equal to or greater than its responsibility to other sites on the internet.

]]> By: Sebastian http://sebastians-pamphlets.com/thou-must-not-steal-the-trailing-slash-from-my-urls/#comment-1271 Sebastian Thu, 06 Mar 2008 00:27:53 +0000 http://sebastians-pamphlets.com/thou-must-not-steal-the-trailing-slash-from-my-urls/#comment-1271 Thanks Tristan for your appreciated thoughts. I totally get your point. I just can't agree. Developers are the middlemen between humans and machines. If a developer fails to translate in any direction that's, well, fault. And yes, the rules are there for a good reason. This good reason is that the Web cannot function without machines, whether they're perfect or not. Machines understand proper syntax and semantics, and they often fail on guesswork. Just because in an ideal world machines would understand intentions and perhaps even read our thoughts, that does not mean that we can throw away the protocols today. As long as we don't live in an ideal world, GIGO applies. Please reread my phone number example. Would you support the civilized human who refuses to make use of the phone number's last digit too? Of course not, because that's plain foolish as long as not all phones on this planet have an autocomplete helper that cannot fail. The "sneakily" was not meant seriously. Thanks Tristan for your appreciated thoughts. I totally get your point. I just can’t agree. Developers are the middlemen between humans and machines. If a developer fails to translate in any direction that’s, well, fault. And yes, the rules are there for a good reason. This good reason is that the Web cannot function without machines, whether they’re perfect or not. Machines understand proper syntax and semantics, and they often fail on guesswork. Just because in an ideal world machines would understand intentions and perhaps even read our thoughts, that does not mean that we can throw away the protocols today. As long as we don’t live in an ideal world, GIGO applies. Please reread my phone number example. Would you support the civilized human who refuses to make use of the phone number’s last digit too? Of course not, because that’s plain foolish as long as not all phones on this planet have an autocomplete helper that cannot fail.

The “sneakily” was not meant seriously.

]]> By: Tristan http://sebastians-pamphlets.com/thou-must-not-steal-the-trailing-slash-from-my-urls/#comment-1270 Tristan Wed, 05 Mar 2008 23:46:44 +0000 http://sebastians-pamphlets.com/thou-must-not-steal-the-trailing-slash-from-my-urls/#comment-1270 It is, in fact, completely different because canonical URIs are a shared syntax between unpredictable human users and machines! That makes it the responsibility of the machine (and the programmers who manage them) to make URIs the most usable they can be for the humans. Humans come first, machines serve humans, it's a general concept. The first step to that is accepting that humans will break the rules, because it's a statistical certainty. They're not "sneaky", they're just human, and we *cannot* enforce how they enter URIs. In fact, we don't want to, because we want to make it as easy and simple as possible, and thus attempt to determine what they want even if they don't follow perfect syntax. It is not their fault and it is not expected of them to be a computer. The problem, then, is that this system is used by both machines and users. That is the problem. The basic, root problem. Like I said before (and that you're not listening to, stuck in your preachy argument) I understand and agree with everything you're saying, that this is the reality, and developers should definitely use correct URI format. I'm simply discussing the whole background of the situation, and if you're not willing to look at why the system is the way it is, and why developers might be accustomed to not caring about the slash, and why the machines care but the users don't -- then you're just spewing rules for the sake of rules. I'm just trying to look at the bigger picture here. Let me just say again, I agree with you completely about the rules and the reality, I just disagree completely on the way you portray them. The rules are *not* there for a good reason, they're there for the machines, ignoring completely the fact that both machines and humans use them and have different expectations and needs. Of course we should follow them as programmers and developers if we know that we should, that's obvious. But you're trying to solve a problem here (that being, that developers use the wrong syntax for URIs) and don't understand the underlying issue (that developers are human users too, and are rightfully accustomed to lax 'human' URI syntax), and that's irking me. Think about it. And one more time, I understand the rules, don't try to explain them to me again. Look deeper. It is, in fact, completely different because canonical URIs are a shared syntax between unpredictable human users and machines! That makes it the responsibility of the machine (and the programmers who manage them) to make URIs the most usable they can be for the humans. Humans come first, machines serve humans, it’s a general concept. The first step to that is accepting that humans will break the rules, because it’s a statistical certainty.

They’re not “sneaky”, they’re just human, and we *cannot* enforce how they enter URIs. In fact, we don’t want to, because we want to make it as easy and simple as possible, and thus attempt to determine what they want even if they don’t follow perfect syntax. It is not their fault and it is not expected of them to be a computer.

The problem, then, is that this system is used by both machines and users. That is the problem. The basic, root problem.

Like I said before (and that you’re not listening to, stuck in your preachy argument) I understand and agree with everything you’re saying, that this is the reality, and developers should definitely use correct URI format.

I’m simply discussing the whole background of the situation, and if you’re not willing to look at why the system is the way it is, and why developers might be accustomed to not caring about the slash, and why the machines care but the users don’t — then you’re just spewing rules for the sake of rules. I’m just trying to look at the bigger picture here.

Let me just say again, I agree with you completely about the rules and the reality, I just disagree completely on the way you portray them.

The rules are *not* there for a good reason, they’re there for the machines, ignoring completely the fact that both machines and humans use them and have different expectations and needs. Of course we should follow them as programmers and developers if we know that we should, that’s obvious. But you’re trying to solve a problem here (that being, that developers use the wrong syntax for URIs) and don’t understand the underlying issue (that developers are human users too, and are rightfully accustomed to lax ‘human’ URI syntax), and that’s irking me. Think about it.

And one more time, I understand the rules, don’t try to explain them to me again. Look deeper.

]]> By: Sebastian http://sebastians-pamphlets.com/thou-must-not-steal-the-trailing-slash-from-my-urls/#comment-1269 Sebastian Wed, 05 Mar 2008 23:20:08 +0000 http://sebastians-pamphlets.com/thou-must-not-steal-the-trailing-slash-from-my-urls/#comment-1269 Tristan, when a user types in a sneakily shortened URI that's fine. When a Web developer --who should know better-- links to a malformed URI that's evil. Yes, that's really evil, because this developer is aware of the not yet perfect machines, and this developer knows that the majority of all machines, including important Web robots like search engine crawlers, do rely on compliance to Web standards. Just because we wish that all this should be easier, that doesn't change the reality. Here and now Web developers just have to comply to the standards, whether they like it or not. Our compilers are not as fault tolerant as Web servers, we understand that so we check the syntax beforehand and don't deliver code that doesn't compile or produces runtime errors when interpreted. Why is that different from sticking to canonical URIs? Tristan, when a user types in a sneakily shortened URI that’s fine. When a Web developer –who should know better– links to a malformed URI that’s evil. Yes, that’s really evil, because this developer is aware of the not yet perfect machines, and this developer knows that the majority of all machines, including important Web robots like search engine crawlers, do rely on compliance to Web standards. Just because we wish that all this should be easier, that doesn’t change the reality. Here and now Web developers just have to comply to the standards, whether they like it or not. Our compilers are not as fault tolerant as Web servers, we understand that so we check the syntax beforehand and don’t deliver code that doesn’t compile or produces runtime errors when interpreted. Why is that different from sticking to canonical URIs?

]]> By: Tristan http://sebastians-pamphlets.com/thou-must-not-steal-the-trailing-slash-from-my-urls/#comment-1268 Tristan Wed, 05 Mar 2008 23:04:15 +0000 http://sebastians-pamphlets.com/thou-must-not-steal-the-trailing-slash-from-my-urls/#comment-1268 And my point being: they should be. Machines should be smart enough to deal with any reasonable input thrown at them. So I'm just trying to say, it's not anyone's fault that we have to work around these limitations, it's just something we have to do. Your essay came across as confrontational toward people who don't understand why URIs should have trailing slashes, when in fact there's no logical reason for anyone to understand that, because it's naturally illogical that the trailing slash matters at all -- except to a machine. I understand it perfectly. I just don't like being told it's my (or any other web developer's) fault, because it's definitely not. It's essentially a flaw in the system that we have to work around, and should be presented as such. And my point being: they should be. Machines should be smart enough to deal with any reasonable input thrown at them.

So I’m just trying to say, it’s not anyone’s fault that we have to work around these limitations, it’s just something we have to do. Your essay came across as confrontational toward people who don’t understand why URIs should have trailing slashes, when in fact there’s no logical reason for anyone to understand that, because it’s naturally illogical that the trailing slash matters at all — except to a machine.

I understand it perfectly. I just don’t like being told it’s my (or any other web developer’s) fault, because it’s definitely not. It’s essentially a flaw in the system that we have to work around, and should be presented as such.

]]>