This post doesn’t have much to do with bit.ly but I want to continue my URL redirector meme and keep mentioning them in the title of my posts as I did here and here.
Twitter yesterday began experimenting with automatically shortening URLs using their t.co URL shorterner. If you type in a long URL and it goes over 12 characters, they will automatically convert it for you. More details from Twitter here.
Twitter’s service is not public. You can’t go to http://t.co and shorten whatever you feel like. It’s an automatically generated thing. They will detect when you are entering in a long URL and then make your life easier for you instead of you having to go offsite, shorten the URL and then copy/paste it back into the Tweet interface.
Yet Twitter is responsible; they will scan the links converted using the service against a list of known malicious sites. That’s a good idea, Twitter is one of the worst hit services for shortened URLs (this is the reason for the title of my post; if a shortening service is known to be a responsible one, should you block emails that contain links to them? They are far more likely to point to legitimate sites than to malicious ones). They are probably one of the prime drivers for the proliferation of malicious redirectors.
One wonders whether or not this is bit.ly’s kiss of death. Bit.ly has diversified and now does things like bit.ly powered redirection (like Wordpress’s http://wp.me), but let’s not overlook the fact that they became popular because Twitter became popular and people needed a way to share long URLs in Tweets where space is limited. If Twitter does it automatically, then that’s a huge part of bit.ly’s original market that makes them redundant (although they would argue that their service does more than shorten, they track statistics and provide vanity domain redirection).
Yet Twitter faces a big problem now that they have their own shortening service – they have to deal with the problem of abuse. Yes, they scan against a known list of malicious URLs, but spammers will create these URLs and then use them to spam (via Twitter) before they are added to the lists. They will now have to specialize in more creative URL analysis techniques like IP and name server reputation, and statistical analysis of users, Tweets and other shenanigans. They do some of this already, but now they have to become experts.
For example, a typical Tweeted URL shows lots of hits and then declines exponentially (especially if a celebrity tweets it). But a spammy URL shows a jagged saw tooth pattern that alternates between hits and declines as the spammer uses various media to spread it around. Twitter will need more sophisticated toolsets - home grown toolsets – to combat abuse. The folks at Twitter are smart but it takes a lot to run the service and make it profitable. How many resources do they want to expend on fighting abuse? On the other hand, they already have the data about abusive users and other patterns. They could combine those to form one heck of a Twitter reputation and analysis protection system, or TRAPS.
Hmm, if Twitter acquires a major abuse problem, will sending email containing links to Tweets get you blocked?
Thankfully, the HTML email notification includes the expanded link URL, as well as the t.co HREF click tracker URL, so many content filters will be able to check the target link against their URI-RBLs without having to walk the 301 redirection.