This URL shortener situation is officially out of control
Sponsored By
I saw a URL today on Twitter to an article on Slate.com. It was a custom short URL - http://slate.me/1h0svt8 but since I was visiting it via Twitter, it was wrapped with Twitter's t.co URL, so I really started at http://t.co/sxSvcJnT2L.
When I visited it for the FIRST time, I got this lovely HTTP interaction. That's SEVEN HTTP 301s, count them, 7, before I get to the destination page.
It would have been 8 redirects if I'd counted t.co as well. Note also that after it bounced around three of Slate's URL shorteners, it also goes through goog.gl as well.
- t.co is twitter's URL shortener that acts as a "safety gate" that allows Twitter to shut down a bad URL at the Twitter level. This means Twitter can stop malware faster, they say.
- trib.al is a URL shortener that provides marketing analytics. They are bouncing me around in order to set marketing cookies because it's the first time they've seen me.
- goo.gl is what you'd think it is, it's Google's URL shortener.
That's a lot of back and forth just to get me a a web page. And getting me a web page is kind of the most important thing the web does. Redirects are being abused and I don't see any work happening in HTTP 2.0 to change it.
The second request to the same URL is better, but still frustratingly indirect.
Every redirect is a one more point of failure, one more domain that can rot, one more server that can go down, one more layer between me and the content.
Oh, and just to be obnoxious, I've created http://hnsl.mn/thisurlisverysmall to make the point. Tweet it!
If you prefer long URLs, you can also get to this post from
http://uniformresourcelocatorelongator.com/aHR0cDovL3d3dy5oYW5zZWxtYW4uY29tL2Jsb2cvVGhpc1VSTFNob3J0ZW5lclNpdHVhdGlvbklzT2ZmaWNpYWxseU91dE9mQ29udHJvbC5hc3B4%7CGOvZCfcZVkHOsSZtVQgwUuzvaJeOVXWmOSgexmjuleGbleSSULshLkwpqddRmAkZYVheOvaZCDELQYCzOZzpvGgrnlGTIHypnqBBwVZEJZzlkpTwazjUBoqJUQJyudWXoBQvIOOfwrmqNpHEZmZXwHTYzrCFFSahOScKYLKAfkroAoopMbjuNcLKwSMEzsGeiawVCedYIZFNNQLWSEvQdkZMkDFyDosVlfLLhGizNOTxGQKHESsLoCoNleNtmlPIXedNEQcomXKRsJKOXzrLiAoJZRctSwYduzqvczTPgBgAjvzXRltkhpYDxkEpgiUqthRNiYgLOiDsFptekhiJxyfMKiCCAHYwNHmuKtlDqanKEZLlyqQYjURVbQQXikPrzUPYKHCmZrPOBUCHGssMyMekyfCkNOwhjgOGSrPPUQYtkLdvLQHjnszIJTOfKKMlBVMUoayZGMigAnIzrRfRtIczCOrhbFWthtMysDnHnTmDaTAfZFUCCGPskHqmVCtGFBhkwGQJxUdgPIYRyQXJJhylmfkCRQrvJqwOqnneWMKDTfcVdRxHXKHPkXoxwVpqfRWpSAzJKWMSKObRjzUfHXINRnlIcEGeigJJEIMcbfyXxMhVDxzwyOMboEfHHzTnQPbLyuRFKLxMlrfQbNMkoKJopCKDnKoxiNSFgBGoWjdYApYozbVIaBLoPHpDHXwgCneURTJdLnfLJPEEtxvmaVchmoNnyCvXnTmYFnFnDnLytKClfNFDJgpYsoyXhvTsuhhYapBzXBcKwacpcsZcMScmJIFHVNnrnxnwPIxHlWDMQKAeljmeInjBjACURODHbCSsgGvaAzaRPCLAhYQmtGYOeUfMqKSGPsEDQLDxSYxvtgGLSrfkOU
What do you think?
Sponsor: Big thanks to Mindscape for joining us and sponsoring the blog feed this week! I discovered Raygun.io and started using it for my side project and I LOVE it. Get notified of your software’s bugs as they happen! Raygun.io has error tracking solutions for every major programming language and platform - Start a free trial in under a minute!
About Scott
Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.
About Newsletter
Hosting By
June 03, 2014 1:41
I can understand Twitter having its own shortened. But it could be made more intelligent - perhaps it could follow links it shortened and points to the first non-30x one it sees.
Alex
7 redirects is a bit over the top - on the up side at least its less than 1k.. well, sort of...!
I have always disliked shorteners because of the lack of transparency, "what" exactly am I clicking on etc...
I have always disliked shorteners because of the lack of transparency, "what" exactly am I clicking on etc...
It's an Inception of links. I assume that Url shortening is automatic in TweetDeck as I can no longer find the functionality.
You used to be able to paste in a link and see it get shortened automagically according to your settings. When I didnt see that happen, I was tempted to use me goo.gl extension on browser bar but refrained..
You used to be able to paste in a link and see it get shortened automagically according to your settings. When I didnt see that happen, I was tempted to use me goo.gl extension on browser bar but refrained..
You ever notice how everyone is making us pay with our time for their information and convenience?
Sometimes it's worth it, but other times it sure isn't.
Oh, and you're always worth it.
:)
Sometimes it's worth it, but other times it sure isn't.
Oh, and you're always worth it.
:)
This is a big problem on stackoverflow. People are editing posts to porn using a shortener: http://meta.stackoverflow.com/questions/257936/auto-expand-shortened-urls
I hate them. They get in the way. I've had issues where t.co breaks for a minute and I can't click any links on twitter.
I often play with the idea of a browser extension that resolves all of these for you when they appear on the page, then caches them. This wouldn't do much good for one person, but sharing that cache among all users could. If there were enough users, the metrics on these redirects would go way down, making them much less useful. Then maybe people would stop using them. Of course, that's just a new single point of failure. If it was hacked, you would have a nice place to inject some bad redirects. Ugh.
For twitter specifically, they solved the problem with the short-sighted solution adopted from the community where there was only the same short-sighted option. That is, instead of adopting an official url shortener, twitter should have allowed links to be added as metadata on a tweet. That way, it takes up no space in the message itself. That's the real solution that they should have adopted.
I often play with the idea of a browser extension that resolves all of these for you when they appear on the page, then caches them. This wouldn't do much good for one person, but sharing that cache among all users could. If there were enough users, the metrics on these redirects would go way down, making them much less useful. Then maybe people would stop using them. Of course, that's just a new single point of failure. If it was hacked, you would have a nice place to inject some bad redirects. Ugh.
For twitter specifically, they solved the problem with the short-sighted solution adopted from the community where there was only the same short-sighted option. That is, instead of adopting an official url shortener, twitter should have allowed links to be added as metadata on a tweet. That way, it takes up no space in the message itself. That's the real solution that they should have adopted.
Hey Scott, have you ever had anyone from the Internet Archive on Hanselminutes? I'm sure Jason Scott would have a lot of information about URL shorteners and other threats to Internet archaeology.
Build a URL redirector that follows URL redirections to their ultimate destination, strips out other redirectors, caches the redirection with a short TTL (like a day or two), and provides the single step redirection.
Would be better to just allow customizing a link's text (like in gmail) everywhere on the web. I would agree that this is not great for transparency, but it's not any worse than a shortened url and it would be more semantic.
Then you could run an analytics campaign and use less characters than a shortened url.
Then you could run an analytics campaign and use less characters than a shortened url.
"Oh, and just to be obnoxious, I've greated http://hnsl.mn/thisurlisverysmall to make the point. Tweet it!"
This would be cute if it were the only time you'd use hnsl.mn, but considering you use it for all of your links it's just plain hypocritical.
This would be cute if it were the only time you'd use hnsl.mn, but considering you use it for all of your links it's just plain hypocritical.
Lets be honest the reason this problem exists at all is because of Twitter. If ever there existed a thing that prevented me from posting a link because of its length its Twitter. Has anyone ever in their life ever verbally told someone a bit.ly url?! You cant because its just as hard as "this slash that slash this slash that" (versus "one or el, or um eye, six, bee, em, en, no en...EN!! NANCY.. 4, 6 9er"). If we didnt live in a retarded world of 140 characters - long urls wouldn't be an issue. But, on top of that, if we didn't have such long insane urls because, you know SEO, then it wouldn't be an issue either.
Martin - Um, no I don't. I haven't used it in a while, and when I do, I do it for a reason. And, it's a DNS link to bit.ly so it doesn't add a redirect.
Ninju - Thanks, fixed.
Adam - I should have him on the show. Good looking out.
Ninju - Thanks, fixed.
Adam - I should have him on the show. Good looking out.
I think its a great point about points of failure. Not to call you out, because we're all guilty of doing it, but some of the links on the earlier hanselminutes podcast are `shrinkster.com/...` -- which are all now broken links.
Yes, the redirection madness has to stop. A lot of times, I'm just interested in the text within the web page being pointed to by the URL shortener. So I let my Google Mobilizer bookmarklet fetch it for me.
Jason, I customize bit.ly links so it's easy to tell them to people like http://bit.ly/Appcelerator for my Titanium presentation, etc.
Hrm, I agree this is a problem, but I'm not sure how HTTP 2 is supposed to solve this particular problem... You going to just hard code a MAX_REDIRECTS value?
@ Ninju perhaps he meant to write "Greated" seems to me that it could be a nice new word describing something that you made that you felt was like, totally rad ;)
You are doing the same thing to outgoing links on this blog.
So why criticise others?
So why criticise others?
There are some "URL longener" services available too (for example: http://longurl.org/). They will check redirects for you. This world is crazy :-)
Well, this mess is mainly thanks to the stupid text length limit in Twitter posts. They could easily fix it simply by not counting the URL length into the character limit.
The malware excuse is lame, they could scan the URLs anyway if they really wanted to do that.
The other reason for these short URLs are mobile phones, but banning the redirectors there would be only beneficial. They are a very common vector for various frauds here - e.g. you get a message saying that you have received an MMS with a link to go read it. That redirects to another page saying you have a voicemail (??) containing another link. And the last one is, surprise, tel://xxxx to a very expensive premium number ...
The malware excuse is lame, they could scan the URLs anyway if they really wanted to do that.
The other reason for these short URLs are mobile phones, but banning the redirectors there would be only beneficial. They are a very common vector for various frauds here - e.g. you get a message saying that you have received an MMS with a link to go read it. That redirects to another page saying you have a voicemail (??) containing another link. And the last one is, surprise, tel://xxxx to a very expensive premium number ...
Well said Scott. I appreciate your island of pragmatism amongst a sea of pretentious hipsterness.
A browser extension could fix that, and would make a great business model. I'm not much into that business, but stealing redirects from all those middle-men seems like gold...
OK, the browser extension would generate traffic by it's own, but it could be worth it ("saving 20 billion redirects every day, and counting...")
OK, the browser extension would generate traffic by it's own, but it could be worth it ("saving 20 billion redirects every day, and counting...")
Another problem with redirectors is that due to their unknown destinations they are often blocked by workplace proxies.
I can't imagine the result when the browser appears to be blocking all redirect directives!
Work may not be happening in HTTP 2.0, but a ton of work went into the DOI (Digital Object Identifier) http://www.doi.org/ initiative going back to 1997. It may seem a little arcane, but there is so much thinking on the topic of persistent, authoritative links.
Well, just think if those servers actually were in the countries their TLDs claimed they were:
Columbia, then three round-trips between Albania and Montenegro, and then to Greenland, with Mongolia and Libya in the mix too.
Columbia, then three round-trips between Albania and Montenegro, and then to Greenland, with Mongolia and Libya in the mix too.
The problem was not created by Twitter, the problem is marketing. Every marketing team wants their metrics. They get them by redirects. Because building your own intelligent data aggregator and display functionality that satisfies marketing does not make your company money. Marketing needs to know what links you clicked, they need to, you don't understand marketing if you don't understand this. They drive me crazy.
Metrics come from a link tracker. Most of these are link tracking. Marketing wants to know what you are clicking.
Metrics come from a link tracker. Most of these are link tracking. Marketing wants to know what you are clicking.
Funny, I was just thinking about this recently. I could tell I was being passed around a couple URLs but hadn't checked how many. On top of this MS twitter links weren't working for me until I noticed that the same link would work from my cell. I then finally realized I had an edited hosts file that blocking one of the tiny URLs
The thing with trib.al though is that it seems like it's more of an analytics tool rather than a url shortener. If we took url shortening out of the equation you'd still get a 4+ 301's for it to set it's cookies and track you. In this case I don't think the url shorteners are the problem, it's the way their tracking service works.
Not only are each one of these is a point of failure, over time one of these services will surely remove the short URL for whatever reason and the link to the article is lost even though the article still exist. Try to visit a link in an old tweet and you'll most likely end up with a 404 at a URL Shortner service.
So what's it about those multiple hosts there?
Do they redirect to the same host several times in a single request?
Do they redirect to the same host several times in a single request?
Even with wordpress on my site www.careerbaba.in when using bit.ly and other similar url shortening plugin, I am always skeptical about this. Its not that difficult and best to write your own custom url shortener. Good job scott.
It seems to me that in this case twitters URL shortener is actually not working ...
How can it protect you from a shortened URL when a URL shortener is effectively abstracting twitter away from the aqctual content.
consider the response if twitter were to check that URL ...
301 "http:// new url"
That's not helping anyone protect against malware!!!
I also wonder if part of the problem in europe is this new EU cookie directive that states you must make a mess of your web page if tracking anyone with a cookie, some of these redirects may just be "some user at IP xxx clicked link yyy" purely to get marketing stats.
Another example of politicians making technical decisions without understanding the problem !!
How can it protect you from a shortened URL when a URL shortener is effectively abstracting twitter away from the aqctual content.
consider the response if twitter were to check that URL ...
301 "http:// new url"
That's not helping anyone protect against malware!!!
I also wonder if part of the problem in europe is this new EU cookie directive that states you must make a mess of your web page if tracking anyone with a cookie, some of these redirects may just be "some user at IP xxx clicked link yyy" purely to get marketing stats.
Another example of politicians making technical decisions without understanding the problem !!
When I clicked on the long link, I was like: "oh nooo, The page is loading within this tab instead of a new one!" then I was like: "oh wait..".
I wonder if URL shortners suffer from a case of unintended consequences. I first learned and used shortners as a means to share content with an insanely long query string. It made emails a little tidier not having a link that spanned three lines.
This handy helper has now been subverted to the dark side.
This handy helper has now been subverted to the dark side.
I feel annoyed by this as well so I googled a bit, found out about longurl.org and its API so I wrote a small chrome extension that allows me to see where the un-shortened url you can check it out here: https://github.com/codrineugeniu/chrome-unshortener
thats why every growing industry needs standard . Here every company is using its own gateway. The suffering is endured by the reader . Good notice scott !! Hope someone will step forward and take the garbage out of it and make it only 1 direct.
I'm going to throw a bomb out there... As a geek, I get the rage about all these re-directs. But, tell me... why is this a problem for my mom? Why should she care about this? And if she shouldn't, then why are we discussing this? :)
I "get" the multiple points of failure aspect. But, if the load performance hit is not noticeable to the average user, who cares?! The marketing machine bean counters are going to count their beans so the content (which we assume has some value) is going to be paid for.
I "get" the multiple points of failure aspect. But, if the load performance hit is not noticeable to the average user, who cares?! The marketing machine bean counters are going to count their beans so the content (which we assume has some value) is going to be paid for.
To easily do what Scott did and un-shorten a URL while tracking how you got there, use unfurlr.com
@Peter
Because she's being trained to click on links without knowing where they lead. And eventually one will lead to malware.
Because she's being trained to click on links without knowing where they lead. And eventually one will lead to malware.
Nice post Scott.
URL Shorteners are quite annoying also if they didn't have any preview functionality.
URL Shorteners are quite annoying also if they didn't have any preview functionality.
Facebook is the worst.. even though it's not shortening anything, it still hang every redirect and too often it just stops :(
The internet is getting worse day by day...
The internet is getting worse day by day...
You got to love this one though, you say so yourself :)
http://shrturl.co/Wzpc4
http://shrturl.co/Wzpc4
Personally, I seriously dislike shortened urls for their lack of destination transparency. One has no idea what is waiting at the end of their often tainted links.
This is not good practice. The use of short URLs per se is fine but this kind of a use of short URLs really works against the overall goal of good page load performance, which all web sites need to be keeping an eye on, especially on mobile platforms. Plus it's just obnoxious. :)
Thanks for finally talking about > This URL shortener situation is
officially out of control - Scott Hanselman < Liked it!
officially out of control - Scott Hanselman < Liked it!
Just realised why my ISP is blocking links from twitter t.co url.
One workaround, in Chrome browser right click the t.co link to show the direct link
Nice post as usual. A (bad/good?) Niche use for URL shorteners is in QR code URLs. A shorter URL gives a less complex QR code. This helps low resolution cameras in smart phones quickly navigate to URLs in the QR code. Problem is unless you personally manage the shortening service, your URL is likely not to be static or even exist forever.
I started writing a custom open source URL shortener called Shorty. I didn't realise you could point your DNS at bitly and wasn't aware of this issue with multiple redirects.
Now I think I will stick with putting the full URL in twitter.
Now I think I will stick with putting the full URL in twitter.
Comments are closed.