Email Servers Will Choke, Says Spamhaus

Rub3X writes, "The legal battle between antispam organization Spamhaus and e360 Insight is heating up. Spamhaus has a user base of around 650 million, and its lists block some fifty billion spam emails per day, according to the project's CEO Steve Linford. Spamhaus CIO Richard Cox says the immediate issue is that if the domain is suspended, the torrent of bulk mail hitting the world's mail servers would cause many of them to fail. More than 90% of of all email is now spam, Cox says, and he doubts that servers worldwide would be able to handle a ten-fold increase in traffic." Others estimate Spamhaus's blocking efficacy as closer to 75%; by this metric spam would increase four-fold, not ten-fold, if Spamhaus went unavailable. The article paraphrases CIO Cox as saying that the service will continue "even if there is a short-term degradation."

kdawson at it again.

[Inoshiro]

Here's the dnscache (part of the djbdns family [wikipedia.org]) solution: /service/dnscache/root/servers# cat spamhaus.org
216.168.28.44
204.69.234.1
204.74.101.1
204.152.184.186
#

No need to HUP -- once the file is created and filled with those IPs, it'll pick them up automatically. You can easily install dnscache with the other tools on your mail servers for 0 interuption of service.

Cheers.

Re:SpamHaus users can use n.n.n.n form URLs

[Anonymous Coward]

Bloody hell.

It is called an IP address.

Spamhaus is correct

[mabu]

Spamhaus is correct in saying that 90% of SMTP traffic on the net is spam. Based on my analysis we're seeing somewhere around 93%. People do not realize how much spam is blocked by relay blacklisting that never even gets to content-based filter systems. Virtually all major ISPs, including AOL, are heavily using relay blacklisting.

If Spamhaus goes down though, ten more RBLs will pop up. It's necessary to stop spam. And they're right... most mail servers on the Internet are not capable of handling the sheer amount of traffic if they were not also hanging up on bogus SMTP connections before even receiving content information. You ever wonder why your e-mail is delayed? This is because your ISP is queing mail processing because they can't handle it all at once. Without relay blacklisting, e-mail would be even slower and likely interrupted. I'm not suggesting that Spamhaus is that important, but what they do in theory, is.

All I can say is, pray that IPv6 doesn't get adopted or it will be even worse.

Re:Use the UK server name!

[Anonymous Coward]

Not at the moment you can't:

> 119.59.126.24.zen.spamhaus.org

Non-authoritative answer:
Name: 119.59.126.24.zen.spamhaus.org
Address: 127.0.0.4

> 119.59.126.24.zen.spamhaus.org.uk

DNS request timed out.
        timeout was 2 seconds.
DNS request timed out.
        timeout was 2 seconds.

Re:I say let the spam come

[.Chndru]

ICANN says no, http://www.icann.org/announcements/announcement-10 oct06.htm [icann.org]

Re:I say let the spam come

[ray-auch]

Actually, the problem (if you read the lawyers who've written on this) is that originally they _did_ go to court.

IIRC they asked the original (state, district ?) court to move the case to federal.

_Then_ they didn't turn up at the federal court because they _then_ decided they didn't accept its jurisdiction.

Re:what else can you do?

[NoSuchGuy]

The latest problem has been with image spams regarding penny stocks. The source shows basically nothing filterable, anyone ever find a way to deal with those?

Use Spamassassin with the "HTML_IMAGE_ONLY_xx" rules

Re:Someone please tell me they have an alternative

[RAMMS+EIN]

``My guess is they'd borrow ideas brought to us by instant messaging. Contact lists, invites, authorizations, etc.''

Spammers now send their messages in MSN and ICQ invites/authorization requests.

Re:I say let the spam come

[cortana]

You're leaving out the part where their solicitors requested the venue change without instructions. AFAIK Spamhaus dismissed them and are taking them to court for creating this whole fucking mess in the first place.

Re:Interesting legal argument.

[91degrees]

It was a commentary on spamhaus' legal argument so far. Not an opinion on how things should work.

1. Spamhaus requested jurisdiction be moved to a federal court in this (PDF) document [e360insight.com], thereby accepting jurisdiction of the court.
2. The Illinois District Court is a general trial court of the US federal court system.
3. Their ciurrent position - after losing horribly through inept legal arguments - seems to be that they're nice people.

Questioning the Math/Assumptions

[carpeweb]

More than 90% of of all email is now spam
Others estimate Spamhaus's blocking efficacy as closer to 75%; by this metric spam would increase four-fold, not ten-fold, if Spamhaus went unavailable

I think the math is a lot more complicated than this implies. Here's how I'd work it:

• P = % Spam (% of all sent mail)
• S(T) = Total Mail Sent
• S(S) = Spam Sent
• S(N) = Non-Spam Sent
• E(T) = Overall Filter Efficiency (% spam detected, Spamhaus + All Other Filters)
• E(S) = Spamhaus Filter Efficiency (% spam detected, Spamhaus Only)
• E(O) = Other Filter Efficiency (% spam detected, All Other Filters w/o Spamhaus)
• F(T) = Overall Type II Error Rate (% false positive, Spamhaus + All Other Filters)
• F(S) = Spamhaus Type II Error Rate (% false positive, Spamhaus Only)
• F(O) = Other Type II Error Rate (% false positive, All Other Filters w/o Spamhaus)
• R(T) = Total Mail Received
• R(S) = Spam Received
• R(N) = Non-Spam Received

We're interested in R(T) and what happens to it with and without Spamhaus. (Assuming we're still interested at all, since math sometimes does that ...).

With Spamhaus:

• R(T) = R(S) + R(N)
• R(T) = S(S) x [1-E(T)] + S(N) x [ 1-F(T)]
• R(T) = P x S(T) x [1-E(T)] + (1-P) x S(T) x [1-F(T)]

Without Spamhaus:

• R(T) = R(S) + R(N)
• R(T) = S(S) x [1-E(O)] + S(N) x [ 1-F(O)]
• R(T) = P x S(O) x [1-E(O)] + (1-P) x S(O) x [1-F(O)]

The difference, expressed as a ratio of (Without Spamhaus - With Spamhaus)/(With Spamhaus), is

[ P x S(O) x [1-E(O)] + (1-P) x S(O) x [1-F(O)] ] - [ P x S(T) x [1-E(T)] + (1-P) x S(T) x [1-F(T)] ]

Divided By

[ P x S(T) x [1-E(T)] + (1-P) x S(T) x [1-F(T)] ]

The assumptions yielding either the ten-fold or the four-fold increase seem to be that E(O)=0, and of course that false positives don't matter. Even with these assumptions, the math in the OP is a bit fuzzy to me:

• E(O) = 0
• E(T) = E(S)
• F(O) = 0
• F(T) = 0 [i.e., F(S) = 0 as well]
  yields (reducing above ratio):
• [ P x S(T) + [ (1-P) x S(T) ] - [ P x S(T) x (1-E(T)) + [ (1-P) x S(T) ] ]
  
  Divided By
  
  [ P x S(T) x (1-E(T)) + [ (1-P) x S(T) ] ]
• Which Reduces To:
  
  P x E(T) / [ 1 - [ P x E(T) ] ]

The ten-fold increase seems to be predicated upon both P=.9 and E(S)=E(T)=1. However, even if that were true, the increase would actually be nine-fold (.9/.1).

The four-fold increase seems to be predicated upon P=.9 and E(S)=E(T)=.75. However, this would yield about a two-fold increase of

[.9 x .75] / [ 1 - (.9 x .75) ] = 27/13 = 2.08 (approx.)

Factoring in false positives might actually make the Without Spamhaus scenario more dire, but clearly it would be less dire if we assume that E(O) is not zero. A better approximation would use the marginal efficiency of Spamhaus. Even with a generous assumption that Spamhaus catches an additional third of all spams sent (vs. all others without Spamhaus, and ignoring false positives), the overall increase in R(T) looks less than 50% to me (.3/.7, or approximately 43%).

Re:I say let the spam come

[kwark]

"Do people really think that the volume of spam received will increase that much?"

Only if Spamhaus is used as the only filtering method. Any decent ISP will have alternatives. Personally I use Spamhaus as the first filtering rule, second in line is greylisting, then clamav and last spamassassin.

95% of all incoming connections to my MTA don't get past greylisting (stupid zombies). Spamassassin catches nearly all spam that was left (also checking sender in a couple dns blacklists).

Without spamhaus the only thing that will happen is that there is a little more mail will be passed to spamassassin and spam formerly in spamhaus will get a lower score (about 1 point in SA 3.0.x).

Re:Use the UK server name!

[PeterBrett]

You missed the point. The GP was simulating an blocklist lookup, whereas you just checked that you could get the IP address for the website. Looking up <suspect IP address>.zen.spamhaus.org returns an IP address (typically 127.0.0.4) if the tested IP is in the list, and unknown domain name otherwise.

Voluntary Subscription Service

[defsdoor]

This is nonsense. Spamhaus is a voluntary list of places you might not want to allow to deliver email to you. The people that subscribe to the list do so out of choice, they can configure their servers to block or score higher (usually) based on a listing in the Spamhaus list. Where in all of this is there place for a Judge, a court or even a whiny little Spam company ? No Judge in the world can force a delisting from Spamhaus. It's no different from me posting a list of companies that I don't like - for whatever reason - and because some people see my list and also decide they aren't going to like them either - being told I must like them. This is bollocks of the most objectionable level.

When are the courts and the politicians going to start serving the people ? Corporations are all about money and self interest - start protecting the populace not the highest bidder.

Re:I say let the spam come

[ArsenneLupin]

What is a gabble?

GP probably meant gavel [wikipedia.org] , the judge's small mallet which he bangs on his table to call for silence or attention.

Re:I say let the spam come

[Anonymous Coward]

I hate spam just like the next guy, but when you make a profitable business from spam fighting, you need at least some clue about how the legal system works.

"The" legal system? You make it sound like you think there's only one. Here's a clue: the US legal system is just one of many legal systems in the world. Spamhaus is based in the UK, where we have a somewhat different legal system. It is not reasonable to expect people based outside the USA to know (or care) how the US legal system works.

Re:Maybe I'm misunderstanding something

[guitaristx]

But aren't these emails ALREADY hitting email servers? It sounds like this speculation is FUD-y.

I mean, it's not like Spamhaus somehow redirects the emails to itself like some sort of Intarweb spam-specific black hole.

As I understand it:
1. Spam is sent by spammer (it's taking bandwidth). Because of how mail packets flow through multiple redundant paths, each mail takes up bandwidth many times its raw packet size.
2. Spam hits email server (it's taking CPU time to process)
3. Email server checks against Spamhaus blacklist (dunno if this is bandwidth, CPU, or both - I'm not terribly familiar if Spamhaus caches that information locally at its client sites)
4. Spam is rejected (taking CPU time)
5. Rejection reply generated/sent (? dunno if it does this; would take CPU+bandwidth both)

So Spamhaus disappears. Yes, it would suck as a email user to get flooded with spam, but would this REALLY cause any more work for the mailservers? I could see (if they are generating rejection replies and sending them) that this might actually be LESS work for CPUs and less bandwidth used.

The way that spamhaus works is by blacklisting IP addresses, not email-specific details of mail coming from those IP addresses. Therefore, email servers can reject the TCP connections from the blacklisted IP address ranges; it is no more complicated (and no more resource-intensive) than IP-address-specific firewall rules. Therefore, the spam messages themselves don't ever get sent and the only bandwidth "wasted" on spam is from the TCP SYN packets that never get a reply.

Re: out of office auto-replies

[King_TJ]

Honestly, the "out of office" autoreply feature (most notably used in MS Outlook) could use some work. For starters, it really needs to be designed so users turning it on are immediately prompted for whether they'd like it to respond to all incoming email, or only to internal corporate mail. Quite often, I've emailed a salesperson at some company, only to get back an auto-reply that's intended only for other employees of his/her business -- not outside customers.

servers choking...

[ninjaz]

First, some stats on the mail server I use from a year ago yesterday and yesterday:

October 15 2005 :

Pieces of spam blocked by realtime blocklists: 9062

Top blocklists:
sbl-xbl.spamhaus.org 7193
bl.spamcop.net 1648
dnsbl.njabl.org 221

October 15 2006:

Pieces of spam blocked by realtime blocklists: 47429

Top blocklists:
sbl-xbl.spamhaus.org 40631
bl.spamcop.net 5240
dnsbl.njabl.org 1558

As spamhaus is currently rejecting 40631 emails which consequently don't have to be processed by spamassassin, it would be definitely be felt on this server were Spamhaus to become available. In fact, the reason I started using RBLs to begin with was due to one of the Spamhaus ROKSO culprits sending about 20,000 messages per hour to a dictionary list of users at a hosted domain. The server was dying then, but using OpenBSD's pf databases together with the spamhaus SBL, the problem was stopped cold.

Re:I say let the spam come

[Anonymous Coward]

Everyone seems to be laboring under the false impression that removing the case to federal court somehow waived the right to object to personal jurisdiction. That is just wrong, and a two minute Lexis search shows how wrong it is. "[F]ederal courts have emphasized that "removal, in itself, does not constitute a waiver of any right to object to lack of personal jurisdiction . . . .") Nationwide Eng'g & Control Sys, Inc., 837 F.2d at 347-48 (citing Wright & Miller)." Hlavac v. DGG Props., 2005 U.S. Dist. LEXIS 6081 (D. Pa. 2005). Even after removing the case, Spamhaus can legitimately contest whether the federal court has personal jurisdiction.

MOD PARENT INSIGHTFUL!!!

[frank_adrian314159]

The bottom line is that the law means something or it doesn't. The decision may not have been the one most sysadmins (or even users) hoped for (and God knows it's not the one I would have wanted), but it was decided within the rules of the law and in accordance of the law as written now. I would hate to think that a judge would make a decision based on what his friends and neighbors might think. This is supposed to be a country of laws. Should it ever not be, that would be a very bad thing.

So stop the judge-bashing. Cases are not supposed to be decided on pragmatic issues when the pragma directly violates previous jurisprudence - legislation is the solution to pragmatics not matching current judicial findings. The bottom line is that Spamhaus f*cked up by not appearing in court. They should have. And, because of that, the judge rendered judgement in a proper fashion. If Spamhaus didn't understand the impact that not showing up in court would have on them (especially if they already had the wherewithal to hire a lawyer to file motions with said court), then they have no one to blame but themselves.

Spamhaus is now free to ignore the court's ruling (they are, of course, based in another country with servers in a third and can do so with relative impunity). The court is also now free to attempt to enforce its judgement in any way it sees fit within the bounds of the law. That's the way the system works. If you don't like it, change the system. Don't bitch at the actors who are merely doing their jobs (and, in fact, appearing to be doing so in an relatively competant way).

Spamhaus is part of the problem

[Anonymous Coward]

Spamhaus should pull the plug. All the way. They are working to prolong something that totally sucks. Shut down Spamhaus. Let email totally tank. Let the traffic explode.

Then grab your torches and pitchforks and go after the freak'n spammers. I'm talk'n heads on pikes in the town square.

That's the way to fix smtp.