Terry Zink's Cyber Security Blog

Discussing Internet security in (mostly) plain English

No, using bit.ly does not get you blocked

No, using bit.ly does not get you blocked

  • Comments 5

Over at Word to the Wise, Steve Atkins has a good post on URL redirectors.  URL shorteners take long URLs and compress them into smaller ones.  However, because bit.ly is the most popular URL shortener (because they originally were the exclusive shortener at Twitter), they are also the most abused by spammers.

The reason that spammers abuse bit.ly is because a bit.ly link is resistant to URL scanning by a content filter.  They are betting that since a spam filter won’t mark a message as spam because of a bit.ly link (due to the volume of false positives), they can send spam messages with URLs to this particular shortener and not have to worry about getting caught by URL filter.

Atkins refutes this line of reasoning:

bit.ly have been on SpamHaus’s radar for quite a while. They’re listed on the SBL multiple times. They’re listed in the DBL – SpamHaus’s newish domain based blacklist, intended for content-based filtering of email. All this means that emails that contain bit.ly URLs are increasingly likely to have serious delivery problems.

This isn’t unique to bit.ly: many other URL shorteners have similar problems – j.mp, su.pr, and others. Nor is it unique to SpamHaus: many other spam filters, public and private, are starting to treat common URL shorteners with suspicion.

Naive use of URL shorteners in your email will send it to the spam folder.

One reason that Spamhaus lists bit.ly on their DBL is because they are seen in so much spam.  However, they are not listed in DBL’s “block” zone but in their “URL shortener” zone.  Their own documentation says that you shouldn’t use that zone to block outright, you should use as a weight in the spam filter.

But even then, using bit.ly as a weight in a content filter will be prone to false positives.  The vast majority of links in bit.ly are legitimate.  It is true that bit.ly is abused and that there are URL shorteners that either are set up for spamming, or don’t do a good job of abuse mitigation, but bit.ly is not among them.  They fight abuse; this is straight off their blog:

The first [line of defense in bit.ly abuse prevention] is VeriSign’s iDefense IP reputation service. The iDefense system is focused on detecting and defeating malware. The iDefense blacklist includes URLs, domains, and IP addresses which host exploits, malicious code, command and control servers, drop sites and other nefarious activity.

The second is the Websense Threatseeker Cloud service, which we’ll be adding to our arsenal of anti-spam tools. Websense will analyze the web content behind bit.ly links in real time, using heuristic tools and reputation data to flag spammy URLs, malicious content and phishing sites.

The third is Sophos, an innovative security service whose behavioral-analysis technology goes beyond blacklists, to proactively detect spam and malware.

Obviously, bit.ly cares about making sure that spammers don’t abuse their service.  They are not the lazy, fly-by-night single-coder type operation that sets up a redirector and doesn’t notice when someone takes advantage of them.

Because of this, a spam filter that decides to block messages with links to bit.ly will be prone to false positives – lots of them.  Bit.ly is the most popular URL shortener.  That’s reality and if you block it, users will complain (especially if you have a global antispam business) and it is not worth the support costs.  Getting users to change their behavior is asking too much because they are accustomed to seeing and using bit.ly in Twitter. 

Blocking mail because it contains a bit.ly link is like the current TSA screening procedures – it’s more trouble than what it is worth. 

Leave a Comment
  • Please add 8 and 7 and type the answer here:
  • Post
  • Bad title for a post, considering there's ample proof that at the moment, using bit.ly links in newsletters causes significant delivery issues.

  • I knew that the post title would be provocative which is why I used it. Is it accurate?  It depends on the filter.

  • "it depsnds on the filter". Then your post is wrong.

    The real fact is that if you use an URL shortener your deliverability is likely to drop. Does it drops 0.1% or 10%? Well it depends on your target, but the only users that will increase their deliverability using an URL shortener are spammers.

    So, I think Atkins post is much more helpful than this one. People sending newsletter should be aware that using an url shortener is not good for them and Attkins explained why.

    You, instead, say: we don't block email including url shortener so you are safe: buyt "you", even if a big provider, are not the whole internet. And guess what? There are many many many servers using blacklists in a way that are not suggested by the list mantainer. So, if some server uses DBL url shortener list as a block list I agree with you that they do a bad thing, but still they do this. So if I don't use the shortener my mail is delivered, if I don't use it my mail is not delivered to them: why should I use the url shortener?

  • Is it accurate? No.

  • I have bit.ly blocked in my host file at this point. I have never seen it used for anything but spam.

Page 1 of 1 (5 items)