Are Spam Blockers Too Strict?

Myrte writes "Wired.com has a long piece on whether spam blockers are blocking wanted messages." From the article: "For years, e-mail users complained that torrents of unwanted messages clogged their inboxes and crimped their productivity. Now, e-mail users, marketers and mailing list operators are more worried that spam filters are blocking out too many wanted messages. AOL isn't the only company to face charges that it improperly blocks legitimate messages. But, as the world's largest ISP for years, it has long borne the brunt of complaints from mass e-mailers over the problem."

I don't understand

[linvir]

it has long borne the brunt of complaints from mass e-mailers over the problem

Does this mean mailing list owners or something? I associate "mass e-mailer" with "spammer", so my first instinct was "You may continue to cry". So are there other mass e-mailers? Does it mean the likes of Amazon? If so they too may continue to cry. I don't need to know about This week's hot deals on Electronics & Photo at Amazon.co.uk.

Not trying to put out famebait but...

[Eric Damron]

Obviously spammers are trying to get through filters by making their email appear legitimate. The closer spam looks like legitimate email traffic the harder it is to block them without also blocking some legitimate email. It's kind of a stupid question with a "WELL DUH!" answer.

Not trying to put out a flame but really guys...

It's not that they're too strict

[Nijika]

It's more that SMTP is too broken. The model we use to communicate with each other is sadly too open, given the potential of the technology for automation. The real solution is to extend or replace SMTP completely.

How is this a "gray area"

[TubeSteak]

A particularly troublesome gray area, Schneider said, involves affiliate marketers. These marketers often send e-mails to people who signed up on a website with whom the affiliate has a marketing agreement. The recipient of the e-mail, however, probably isn't aware of the arrangement and has no idea why they're receiving the message.

Translation: people are getting e-mails they neither want, nor expected.

It's like inviting someone to a party & you agree that they can bring their "affiliates" along. Your invitee shows up with 20 strangers & whoever you have working the door says "I don't know all these people, they aren't allowed in."

The solution isn't to cry about the "gray" area, it's to explicitly tell people who the fark these affiliates are & what they'll be sending.

Confirmation challenge

[Spazmania]

When I get a message with a moderate probability of being spam, my spam blocker sends a message back requesting that the sender confirm the message. Works great. Those few legitimate senders stuck on a problematic server can still get their messages to me and so far no spammer has attempted to bypass it.

The only time it doesn't work is when the sender's spam blocker dumps the confirmation request or when the sender doesn't understand what to do.

Don't send mass e-mails

[iamacat]

Just like door to door salesmen and tele-marketers, mass e-mailers have ruined their reputation as a group and are no longer effective at what they are trying to do. If you want to keep your customers updated, offer an RSS feed, personalized with their user id if necessary. Times change, deal with it.

Yes and no

[Bogtha]

If a user has signed up for a mailing list, and doesn't get what they asked for, then that's a false positive, no matter how commercial the mailing list. And this does happen. So in that respect, spam blockers are too strict.

But on the other hand, I fish out a few false positives from my spam dump every month and look to see why they were blocked. In most of the cases, it's because the mailing list operator is doing something dumb. For instance, the last false positive I received - for a legitimate, informative mailing list I deliberately signed up for - triggered my spam filter because of forged headers, two counts of malformed headers, and every other line was in all caps.

The reason why they were caught out was because they used what appears to be a mass mailer designed for sleazy purposes, and they didn't bother with any QA.

Anybody who is running a mailing list should follow a few simple rules:

1. If you outsource, outsource to a reputable company.
2. If you run the mailing list yourself, use reputable software.
3. Set up an email account for every popular spam blocker, and include those addresses in your mailing lists. Check those accounts every time you send out an email, to see if you are blocked by any of them.
4. Never buy email addresses. Ever.

That's what I consider to be common sense, but apparently common sense is hard to come by these days.

Re:Don't send mass e-mails

[c0d3h4x0r]

Your point is actually true in a more general sense.

In general, if people want something, they will seek it out for themselves.

People don't want or need to be advertised at in any way via any means. This applies to companies trying to sell products or services, religions trying to amass followers, or political activists trying to rally voters. It's all BS.

If I want something, I'll go seek it out for myself. Leave me the hell alone. It's not your place to constantly bother me.

SMTP is brain dead and should have never been used

[postbigbang]

This is what happens when you don't think forward on protocols. The cure, in the form of hundreds of attempts at everything from Baysien filters to source-IP blockers, seem to always fail. Why? Because SMTP, our mail protocol, is based on telnet, 7-bit ASCII, and easily fudged authentication. Worse, 'thinking' filtration systems use a rules basis that appears to work, but can never work because the rules can change, as any successful spammer knows.

Then, we get a bunch of techno-idiots like the US Congress to legislate email relationships, miserably, contributing further to the problem.

The real solution? Simple blockage. Route the bastards to 127.0.0.1. Force authentication of the address and its owner before it can go out of the blocked ACLs. And if it happens again, shunt the address to a different CIDR block. Or re-write SMTP. That's all that's going to work. Nothing is foolproof because fools are so ingenious. Never underestimate the power of a hacker, and locks keep your friends out, your enemies have pick tools.

Re:Not a chance

[statusbar]

The real problem is that people are typically assuming that email is a reliable and secure technology, when it is not at all. People just need to learn about using 'return receipts'. The alternative is to use an entirely different communications protocol for messaging.

--jeffk++

Should be a given

[SPaReK]

This should be a given. If you try to block spam, you are going to block some legitimate messages. Hopefully, your ratio of blocking spam messages against legitimate messages is good, but it will never be perfect. This is due partly because spam itself is subjective. A lot of spam messages can be picked out and determined to be a spam message by 10 out of every 10 people. But for some messages, its not that simple. It's just real subjective. Then you're asking an algorithm to use subjective logic to determine whether a message is spam or not and problems just occur. Like I said, for the most part these filters work pretty good, but its not going to be perfect and anyone that thinks so, is just not thinking straight.

I am not opposed to some degree of flagging an alleged spam message, but to discard it without the end user knowing about it is where issues begin to arise. By flagging a message, the end user is able to use their own discretion to determine whether a message is a spam message and they can do whatever they want with those messages.

This isn't to say that RBLs and spamlists are a bad idea, just if you implement one of these, then be prepared for some type of backlash. Perhaps in some cases an RBL is necessary, but to think that using an RBL you are going to stop all spam and all of your clients are going to be happy, that's just wrong.

Re:It's not that they're too strict

[hackstraw]

The real solution is to extend or replace SMTP completely.

People say this from time to time, but they conclude that its still best the way it is. I value mailing lists, and making people pay or whatever proposed mechanism there is simply does not cut it.

I get spam sent via email. I get spam in my snail mailbox. I get spam on my fax machine. I get spammed by cold calls from sales drones/marketers. I've never had this happen (yet), but I've seen someone's phone get spammed with hundreds of porn text messages over a 10 or 15 minute time period. The user was initially billed for the porn spams and had to call the phone company to get them taken off of there bill.

It just seems as though open communication is just going to be subject to spam. Don't want it? Use your own private network to communicate.

Re:SMTP is brain dead and should have never been u

[chill]

Force authentication of the address and its owner before it can go out of the blocked ACLs.

This would be so trivial to bust thru and automate it isn't funny. What happens to zombie machines? They can authenticate fine, so slip right by this problem. Instead of sending thousands of messages as fast as possible, use thousands of zombies and send just and handful messages each. You'll never trip the thresholds for volume and the spam will be buried in among the legitimate e-mail sent by that user.

Authentication is not a solution.

Re:I don't understand

[Anonymous Coward]

>> I associate "mass e-mailer" with "spammer"

That's an invalid assumption.

People sign up for newsletters. There are 300,000+ who've subscribed to ServerSide, for example (mostly Java developers). That's mass e-mailing.

Re:It's not that they're too strict

[Vancorps]

Seems to me its easier just to use domainkeys and senderID. The problem is standardizing. I can't require either one of them because not enough people are compliant. When that changes the spam world will get simpler until a flaw in the mechanism is found which I believe will lead to an encryption war.

One word

[Anonymous Coward]

WHITELIST. If you want it, whitelist it. If you don't have it whitelisted, then the SPAM filter can classify it... If it does it improperly, then tell the filter that it is/isn't spam (as the case may be).

Teach the users how to do this, and let the whiners kill themselves with angst.

Oh please

[dereference]

Ok, this...

If I want something, I'll go seek it out for myself. Leave me the hell alone. It's not your place to constantly bother me.

...does not imply this...

In general, if people want something, they will seek it out for themselves.

...unless you happen to be the sole embodiment of every consumer in the world. See Hasty Generalization [wikipedia.org] for more details.

Look, I'm with you. I hate this stuff as much as you. It's usually even a nice safe rant for a few insightful mods, but yours is practically a troll.

I can assure you that there are quite a few hundred thousand consumers out there who do not share our outlook on this subject, who become very hostile when you fail to keep them informed of important information, and who couldn't set up an RSS reader if their lives depended on it.

Sorry, I'd love to live in that fantasy world, but you have to face that it's just not reflective of reality.

Re:Confirmation challenge

[Josh Triplett]

Or when you spam all the people spammers use as their forged From addresses.

SILENT spam-blocking is the worst kind

[billstewart]

Email became a reliable tool when everybody pretty much accepted the policy that you either deliver the message or hand a rejection to the sender, or at the very worst case, if you've accepted the mail for delivery and can't deliver it, you send a reject message. That was especially critical for UUCP mail before we had the commerial Internet, but it's still critical today.

AOL is rumored to do most of its spam-blocking without notification to the sender or recipient, and that's a big problem and they're hardly alone in this behaviour.

If there's anything broken about SMTP's handling of spam, it's that you sometimes don't decide that a message is spam until after you've accepted it, so it's hard to provide synchronous notification in case it wasn't spam. (SMTP milters let you look at the message body and run it through spam filters before accepting the message if you want to do that, but a message might already be sitting in the recipient's mailbox before you figure out that 1000 of your users have received identical mail and 99 of the first 100 users that read it marked it as spam.)

Re:Confirmation challenge -- Thank you so much!

[alexo]

> When I get a message with a moderate probability of being spam, my
> spam blocker sends a message back requesting that the sender confirm the
> message. Works great. Those few legitimate senders stuck on a
> problematic server can still get their messages to me and so far no
> spammer has attempted to bypass it.

Well thank you so much!

Since the lowlifes started forging "from" addresses using my domain, I am getting several such "confirmation" messages every day. And while my spam filter is doing its job pretty well, I have not found a way to filter out your smug verifications without getting rid of the legitimate ones.

So, thanks to people like you, I get 5 times more verification requests than actual spam.

You better hope that there is no higher power because if there is, and it decides to grant my wishes just when I get yet another verification, you'll have a bit of a problem removing that sequoia from your rear orifice.

Re:Spam blockers ruined my life.

[Auntie Virus]

As the only IT guy of a company that has million dollar clients, I can assure you, all the important client domains are whitelisted. But still there's bound to be some asshat VP of some company who sends something important from a numbered friggin Hotmail account....