The Next Step in Fighting Spam: Greylisting

Evan Harris writes "I've just published a paper on a new and unique spam blocking method called "Greylisting". The best thing about it other than achieving better than 97% effectiveness in blocking spam, is that it practically eliminates the main problem of other solutions: the false-positive. There's even source code for an example implementation written as a perl filter for sendmail, along with instructions for installing, so you can get up and running quickly."

your first mistake

I'm going to try to say this as nicely as possible and without trolling:
You have just rendered Greylisting pretty useless by making it open source. Spammers are much smarter than you think and what you have basically done is shown them what they need to do in order to get around Greylisting. That's just my take on the issue, maybe I'm wrong but I doubt it.

security through obscurity, again?

If they can get around it by looking at the source, then something was wrong with it, waiting to be exploited. Might as well fix it.

Re:security through obscurity, again?

The thing that is wrong is the SMTP protocol, and most people's conception of a spammer. Once you see a few "confessions of ex-spammers", everything changes.

There are people out there who pay $10000 in startup costs, and then make $2000/week for spamming. The $10000 gets them software written by knowledgable internet security experts. This software finds any and every way to anonymify the email spam, and finds lists of people to spam.

As long as knowledgable internet security experts are getting paid good cash to enable spammers, and SMTP doesn't change, spam will only continue to get worse. There needs to be a fundamental change in SMTP protocols. It oughta take the spammers about 2 days to fix their MTA bug to get around greylisting.

Re:security through obscurity, again?

The way to get around this, of course, being that you send each email twice. In other words, run through your database, then run through your database. Same IP addy, same sender, same recipient. As far as the MTA's concerned, it's retrying. Boom.

Re:security through obscurity, again?

I see that, in fine /. tradition, you didn't RTFA.

From the article: If we have never seen this triplet before, then refuse this delivery and any others that may come within a certain period of time with a temporary failure. (emphasis addded)

Later in the article it goes into much more detail about the delay, how long to delay if the triplet has not been seen before, life time of the whitelist, etc.

It also talks about configuring the times - they mention the default delay is 1 hour, but that their records suggest that 1 minute would have caught 99% of the same spam messages - "The data collected during testing showed that more than 99% of the mail that was blocked with the tested setting of 1 hour would still have been blocked with a delay setting of only 1 minute. At that point, having a larger initial delay will definitely help, as it gives time for other blocking methods to act. For this reason, it is suggested that at least a one hour delay value be kept as a default, since spammers will start adapting as soon as this method becomes known and starts being used. (again, emphasis added)

RTFA!

Re:security through obscurity, again?

is reject the mails on the greylist after holding the connection for, say, 10 minutes. That will help deter spamming software,

I doubt it. I would assume the spam software would have a timeout, and I doubt it's ten minutes. If they want to hit-and-run and aren't even willing to make a second delivery attempt when an error code is returned, I doubt they're going to wait 10 minutes. I'm sure that within 30 seconds or less they'll consider it a dead connection and hang up.

Problem is, I used to have my sendmail HANG UP in real-time on an incoming connection as soon as it realized a message was spam. I.e., the incoming message was filtered in the DATA phase and if it was spam I hung up immediately. It worked great and it felt good, but there were many spam programs that took the disconnection as some kind of TCP/IP failure and immediatelty tried again. So I had one day where a single message was attempted to be delivered about 30,000 times as the spammer connected, I hung up, spammer software said "Oops, let me try again!" About one delivery attempt every second or so.

I'd be willing to bet if you put a 10 minute timeout in sendmail you'll see lots of spammer software disconnecting sooner and just trying again. It takes more of their resources, but takes more of yours, too.

Re:security through obscurity, again?

RTFA!

There is no magical waiting period or re-try period that cannot be trivially coded around. And, with good money on the line, will be trivially coded around.

You don't get it. Really smart people are getting paid a whole lot of money to make programs to exploit every possible crack in the way we send email. There is no general rule to spammers, except that it is a lot of money and they are very clever. Little bandaids are not going to stop this one - there needs to be a much more fundamental change. And I am not talking about laws against spam - I am talking about changes in the protocols we use to send email.

Re:security through obscurity, again?

There is no magical waiting period or re-try period that cannot be trivially coded around. And, with good money on the line, will be trivially coded around. You don't get it. Really smart people are getting paid a whole lot of money to make programs to exploit every possible crack in the way we send email.

Yeah, spammers are so clever. Well, the fact is if for every one of these "smart" (yeah, right) spammers who has the help of a network consultant that will work around greylisting there are 5 dumbasses who don't (and I think I'm being generous there), then if I greylist I'd think over 80% of my spam problem would be eliminated. What's wrong with that? What's to "get"? Looking through headers I see the same bulk mailers used over the years, probably passed around as warez in spammer circles.

Re:security through obscurity, again?

As stated, the only reason the hour works right now is because the spammers don't see this in the wild. Re-running your database script an hour later isn't a big deal.

I disagree. When you are sending 250,000,000 emails a day -- restarting that script IS a big deal. It would, in effect, make them have to do the entire thing twice. That's a pretty big hit on their resources.

Re:your first mistake

I'm going to try to say this as nicely as possible and without trolling:

Not trolling at all - you have a legitimate (though perhaps misguided) problem with this method.

You have just rendered Greylisting pretty useless by making it open source. Spammers are much smarter than you think and what you have basically done is shown them what they need to do in order to get around Greylisting. That's just my take on the issue, maybe I'm wrong but I doubt it.

So, the spammers themselves will be of significant help in debugging and helping to fix the code so they can't circumvent it, won't they? OSS means anyone who finds how the greylist script is beaten can figure out a fix and post it. Sounds like the best thing to do IMHO.

Soko

Re:your first mistake

You have just rendered Greylisting pretty useless by making it open source.

You're assuming the spammers can read source code.

Re:your first mistake

You're assuming the spammers can read.

Re:your first mistake

You're assuming the spammers can read source code.

Who do you think writes spamming software?

Re:your first mistake

That's just my take on the issue, maybe I'm wrong but I doubt it.

That's what I like to see. Someone with strong opinions. Or maybe not.

I think not

Doels this mean all public crypto algorithims are useless?

Re:your first mistake

You're missing a big part of it though. If you have to try say 3 times to send a message [over a 5 day period or so] you're ability to mass send 100million emails is really squashed.

Legitimate people first time sending won't really mind the few day wait and most MTAs will try for upto a month.

Tom

Re:your first mistake

not at all

Read the paper. Spammers would figure it out eventually. What it buys is what they have to do to get around it.

It means they have to do retrys...that means spam runs take longer, especially since they have to run...then wait for a locally defined timeout, and run all those addresses again

AND they have to do it from the same IP.

This raises their bandwidth profile. It wastes their time... all in all... it raises their cost of doing buisness and cuts into their profit margins.

It means they will have to upgrade their tools again. It means they get headaches. And of course, the next step is to impliment spam traps that watch activity and see that a spammer is spamming, and promotes them to a blacklist before they can even retry. (oh gee 1000 new greylist triplets from 1 IP in under 5 mins? Set the timeouts for that IP to 12 hours)

-Steve

Re:your first mistake

It means they have to do retrys...that means spam runs take longer, especially since they have to run...then wait for a locally defined timeout, and run all those addresses again

AND they have to do it from the same IP.

Not to mention that if this is used in conjunction with other collaborative tools (i.e. RBL, checksums), by the time that the spamming MTA can return its IP address will have been submitted to MAPS/etc. and the contents of the message will have been submitted to Razor/Pyzor/DCC.

I think that this greylisting idea will be pretty hard to beat by Joe spammer. Since the game of spam detection is pretty much an arms race, slowing him down will probably be enough to turn the battle in your favour.

Re:Questions

1 hour is the time proposed. Completely unacceptable unless the whitelist works.

Since most personal users are on dialup or dynamic IPs, unless the mail client can upload the whitelist in a trusted fashion (or the MTA remembers what users the client sent messages to!), this won't work.

Do any mail clients include whitelist-collection? Mail.app for OS X does collect all addresses you've sent to, but I've never seen any tool to upload it somewhere.

Re:Questions

It's not the sender's mail client that connects to the server that runs the greylist system, it's the sender's SMTP server as provided by their company or ISP. Its IP address will not change regardless of the sender's connection or dynamic IP.

can't believe their numbers

most spam today is sent through open relays. Those relays will simply retry the delivery no matter which software the spammer uses, so the method won't work.

Re:can't believe their numbers

Eh, open relays are soooo 20th century. :) Actually most open relays today are either blocked or closed, and newly installed MTAs are secure against third-party relaying by default, so this spam method is dying out [it-analysis.com]. Most spam today is sent either directly to the receiving MTA, through open proxies, or through formmail.pl and similar exploits.

Poor use of statistics

The data in this article claims that 1% of all corporate mail servers in the UK allow open relaying, down from 91% in 1997. For all we know, the total number of corporate e-mail servers has grown by a factor of 100 (or more) in the last six year, meaning that perhaps there are more open relays now.

The article also doesn't measure the amount of spam coming through those relays. Even if there are only 10 open relays in the UK at any one time, it still might be possible for all of the spam to be coming through them.

Certainly, closing down open relays is a good thing, but lowering the percentage of open relays doesn't prove anything about the source of spam

In case of /.'ing

The Next Step in the Spam Control War: Greylisting
By Evan Harris
Copyright 2003, all rights reserved.

Introduction
This paper proposes a new and currently very effective method of enhancing the abilities of mail systems to limit the amount of spam that they recieve and deliver to their users. For the purposes of this paper, we will call this new method "Greylisting". The reason for choosing this name should become obvious as we progress.

Greylisting has been designed from the start to satisfy certain criteria:

1. Have minimal impact on users
2. Limit spammers ability to circumvent the blocking
3. Require minimal maintenance at both the user and administrator level

User-level spam blocking, while somewhat effective has a few key drawbacks that make its use in the continuing spam war undesirable. A few of these are:

1. It provides no notice to the senders of legitimate email that is falsely identified as spam.
2. It places most of the costs of processing the spam on the receivers side rather than the spammers side.
3. It provides no real disincentive to spammers to stop wasting our time and resources.

As a result, Greylisting is designed to be implemented at the MTA level, where we can cause the spammers the most amount of grief.

For the purposes of evaluating and testing Greylisting, an example implementation has been written of a filter that runs at the MTA (Message Transfer Agent) level. The source for this example implementation is available as a link below, and as other implementations or additional utility code become available, they will also be linked.

Greylisting has been tested on a few small scale mail hosts (less than 100 users, though with a fairly diverse set of senders from all over the world, and volumes over 10,000 email attempts a day), however it is designed to be scalable, as well as low impact to both administrators and users, and should be acceptable for use on a wide range of systems, including those of very large scale. Of course, performance issues are very dependent on implementation details.

The Greylisting method proposed in this paper is a complimentary method to other existing and yet-to-be-designed spam control systems, and is not intended as a replacement for those other methods. In fact, it is expected that spammers will eventually try to minimise the effectiveness of this method of blocking, and Greylisting is designed to limit options available to the spammer when attempting to do so.

The great thing about Greylisting is that the only methods of circumventing it will only make other spam control techniques just that much more effective (primarily DNS and other methods of blacklisting based on IP address) even after this adaptation by the spammers has occurred.

The Greylisting Method
High Level Overview
Greylisting got it's name because it is kind of a cross between black- and white-listing, with mostly automatic maintenance. A key element of the Greylisting method is this automatic maintenance.

The Greylisting method is very simple. It only looks at three pieces of information (which we will refer to as a "triplet" from now on) about any particular mail delivery attempt:

1. The IP address of the host attempting the delivery
2. The envelope sender address
3. The envelope recipient address

From this, we now have a unique triplet for identifying a mail "relationship". With this data, we simply follow a basic rule, which is:

If we have never seen this triplet before, then refuse this delivery and any others that may come within a certain period of time with a temporary failure.

Since SMTP is considered an unreliable transport, the possibility of temporary failures is built into the core spec (see RFC 821). As such, any well behaved message transfer agent (MTA) should attempt retries if given an appropriate temporary failure code for a delivery attempt (see below for discussion of issues concerning non-conforming MTA's)

Tempfailing is not new and unique

This idea isn't so new or unique. It's been discussed a fair bit on the ASRG [ietf.org] mailing list under the name "tempfailing".

First I heard of it was from Landon Noll and Mel Pleasant. It is noted in brief as one of the techniques in this plan to end spam [templetons.com] (though their plan, which did include the triplets, is not laid out in full there.)

It is a worthwhile technique for a little while, and if spammers were rational, would be worthwhile for some time to come. But spammers are not rational, and already this technique is not as useful as would be hoped.

Do a Google Search for Tempfailing [google.com] especially in ASRG to see statistics etc.

Easy way to stop spam...

Just encode your e-mail address on web pages & don't sign up to any dubious mailing lists.

I haven't received 1 single spam in recent months from doing this!

Easy for end-users, sure.

Just encode your e-mail address on web pages & don't sign up to any dubious mailing lists.

Many of us must maintain contact addresses in the global whois database - so that people can contact us when something is broken.

Look at it this way: you can stop crank calls by unlisting your phone numbers. But you can't unlist the hospital, the ambulance service, the fire department, etc.

We're not all end-users. Some of us are the plumbers.

clever hack for WHOIS contact addresses

The registrar I use (jumpdomain.com) has a clever hack for despamming WHOIS contact email. Basically they change your published contact address once a week. The published address i automatically generated, looks like gibberish, and forwards to your real address. If someone wants to contact you by looking up your address by WHOIS and writing to you, it works fine. But if they add the address to a mailing list, it stops working in a week. That has eliminated almost all my WHOIS spam. Good scheme.

1 false positive is not acceptable.

This isn't very reassuring:

"it practically eliminates the main problem of other solutions: the false-positive."

What does 'practically eliminates' mean? If it gives false positives at all, it is just as useless as all those 'other solutions'.

Re:1 false positive is not acceptable.

Wrong. 1 false positive can be acceptable, and in fact is probably better than how things are now.

At USENIX '03 there was a paper presented on artificial intelligence techniques for spam detection. I can't provide a link since only USENIX members can download the paper (at this point, at least). I was a coauthor of that paper.

One of the things we've discovered in our research is that some classes of filters (most notably, the one I have been developing along with a few other individuals) are actually more effective at correctly classifying email than humans are. That is to say, you can train the learning algorithm on mostly-correctly-classified data, then re-run it over the training data, and almost miraculously, it discovers all kinds of email in the training set that was incorrectly classified.

I.e., this filter has discovered mail that I myself incorrectly thought was spam. It's scary, because there's a lot of it.

To assume that a human will always be 100% accurate at classifying their own email isn't just arrogant, it's plain wrong. Newer filters that will be introduced in the near future might possibly be more accurate than you, a frail human, could ever be.

Re:1 false positive is not acceptable.

Maybe we don't want them to be so accurate.

I get these chain emails from my brother. They are always some funky scheme to get money that won't work. I'd love to just delete them...but if I do this, he tells my mom I don't answer his email.

She then laces into me like you would not believe...blah blah blah he's your brother and you should love him. I don't need that grief...so instead I respond with a "not interested, no cash right now." Keeps the family happy.

I could see it being more important than this, though. Your boss sends you direct mail HE received and appends a "Should we do this" to the bottom. Or, worse, your marketting team constructs a direct mailing that fails your spam filter (no comments from the peanut gallery...obviously this is a good thing to find out, but this is not the way to find it out). Missing that one email could make somebody VERY angry and put you in danger. I have had messages from my boss/CEO/etc go into my junk folder and found them when cleaning it out.

It is correct for the spam engine to label these as spam email. It would be incorrect for it to delete them before they got to you. And so I subscribe to the school of thought that a single false positive makes any spam filter absolutely worthless. It is very easy to delete a message that gets through the filter. It is impossible to resurrect a mailing you never even knew you got.

Time critical

Time critical mailing will go out the window. I can see how this might make any corporate user irate. The same thing goes for challenge-response, the time delay in the business world is unacceptable.

This would be great for personal mail, but that's about it. ISPs would have the same problems with it because their business-class users most likely use the same servers as their consumer-class users.

Re:Time critical

This isn't true, actually. Once one mail gets through, the system lets in subsequent mails from that sender. So there is only the initial delay, after that CEO Joe can use his email as a fat instant messenger per usual.

Re:Time critical

How often do you get time critical e-mail from someone you've never recieved e-mail from before?

some guy telling you to BUY THIS NOW != time critical.

your wife telling you to BUY THIS NOW == time critical, and in theory, your wife == whitelisted (or blacklisted, depending on personal preference).

spam.....hrmmm

with all of these solutions to spam..and all of the spam now flooding mail servers...

isn't it time to change the specification (RFC) and possibly the manner in which our current system works? i haven't come up with anything yet, but surely there must be some sort of handshaking/secure type connection that could be used - - some sort of postage (free) that is encrypted into the mail, that states that it is genuine....kind of like the hologram on those windows cds...

i dunno. file this story under redundant.

How about Habeas' haiku method?

The best idea I've seen in YEARS was to have people start using a specific, original poem as their signatures. Then, the author granted license to anyone who WASN'T sending spam. Therefore, they could sue any spammer for copyright infringement if they used it, and you could train your mail filter to look for the signature. Once spamassassin took it up, it pretty much snowballed. See story here [wired.com]

I'm not sure about this...

These applications appear to adopt the "fire-and-forget" methodology

I thought it was generally understood that most spam was sent by abusing open relays, thus hiding it's origin. This could be wrong. However if it's not, those figures aren't appllicable. Nor is spam going to be diverted since an open relay is generally running a regular mta and will attempt a retry. For instance, if qmail were running on an open relay and was abused by a spammer it would try again and again with an increasing delay (calculated logarithmically if memory serves) between attempts. So the mail will still get through.

When you further consider that if a spammer hits an open relay and hammers your mailserver from it and all of the "triplet's" are new, you're increasing your traffic, because all of that mail will be attempted again.

Bayesian Filtering

I'm finding that use of the Outclass interface to POPfile is surprisingly effective at dealing with my spam problem (and I get a lot of it) - since training POPfile I haven't had a single spam message get into my inbox no false positives. Of course I could just be very, very lucky and with this post the email gods will punish me...

How does the effectiveness of Greylisting compare with what others are seeing with existing techniques (such as Bayesian filtering)? Is it a false positives problem, such as digests and opt-in mailing lists getting incorrectly tagged as spam?

Re:Bayesian Filtering

The key is to make spammers not make money!

If people start adopting anti-spam technologies we would reduce the return spammers get from sending spam. Reduce this enough and the spamming business will no longer be profitable.

POPFile is great. I've also used SAProxy (http://saproxy.bloomba.com/) under windows and it works great too.

Again, the idea is not to eliminate all spam, but to reduce the return rate, and therefore the money made by spammers.

RFC 3514

How about in the spirit of RFC 3514 (the evil bit) we create a spam header in email. Spam will set this header so we can easily filter it out.

Published a paper?

Where? To me, publishing a paper means your writing appeared in some peer-reviewed journal (where the "peers" are acknowledged as domain experts). What you did was put up a web page. With a donation link at the bottom.

For others looking for a solution, try POPFile [sourceforge.net]. Open source, cross platform, gives me 96% accuracy.

One more thing: "practically eliminates" is not the same as "eliminates".

Re:Published a paper?

To me publishing a paper in a peer reviewed journal instead of on the web would mean that I'd expect audience to be reduced to a ridiculously small fraction of people that might be interested. If I wanted to publish something I'd do it on the web first, and if it stacks up people I respect would start talking about it and link to it.

Yes, I realize that for "serious" science still expect things to be published in peer reviewed journals, but in most cases I can't help but think that getting the article out there would be more useful. Sure, peer review is important, and somewhere to look for some kind of verification of the value of a paper is useful. But I much prefer the Research Index [researchindex.com] way, where I can get a good indication of the value of a paper by looking at how many people have cited a paper and WHO have cited a paper.

Anyway, pretending that putting up a document on a website is somehow less publishing a paper than having it printed in a journal, is just plain elitist. You should propably be a bit more critical to papers that are published that you don't know have been through a proper review, especially if you're not a domain expert yourself, but being aware of the source is something that you always need to be.

Waiting for Article Title

The Next Step in Fighting Spam: Death Penalty

Delaying email by one hour!

From the linked paper:

An hour is short enough that in most cases, users will not notice the delay.

I'm wondering how I'm going to explain that to a new customer over the phone who says "I'll just email that file right now so we can go over it together".

Re:Delaying email by one hour!

Agreed. I've been involed in operating a larger (hundreds of thousands of active users) mail system a couple of years ago, and users would complain if their mail took more than seconds. We had to upgrade our system at one point because rapid growth had made mail delivery take a couple of minutes on average, and it caused bad publicity - a lot of users had a clear expectation that e-mail should be delivered in a few seconds and that if it didn't something was wrong.

I think changing that perception of e-mail as near instant will be incredibly hard. And if you succeed it will just move even more traffic over to the IM networks and cause spamming of IM networks to escalate instead.

Re:Delaying email by one hour!

Saddly, you have missed the central point about the necessity of timeliness of email delivery and instead focused on using FTP rather than attachments.

Even if FTP were a solution, it does nothing to answer a new customer who says "I just heard about you and I'm excited about your products. Wanted to call and ask you some questions. I sent an email about 10 minutes ago with an outline of the project we're doing were you guys could really help out, have you had a chance to look it over yet".

There's a limitless number of these important common customer relationship scenarios, where the expectation of all parties involved is that email is delivered in under 1 minute and typically 5-10 seconds. And there are an infinite number of scenarios other than sales and customer service/relations where people quite reasonably expect email to be delivered in seconds.

Focusing on using FTP isn't just the wrong answer, it's not even an answer at all to the problem of email delivery taking an order of magnitude longer than users expect and depend upon.

But as others have pointed out, most users don't have access to FTP servers to receive files. Most corporate firewalls would prohibit users from setting up a FTP server. I would guess that almost any employee behind a corporate firewall wanting to somehow receive a file from a new customer via FTP who attempted to ask a sysadmin would get the answer "just have them send it as an attachment". FTP is simply not a viable protocol for customers and salespeople (or most others) to use to pass files back and forth.

Aside from not solving the unacceptable delay and the inappropriateness of using FTP, there is the problem of bad attitude. Specifically:

Explain this to your

user. You can just tell them that... [snip]

Where did "new customer" turn into "user". The word "user" in this context is often spoken in the tone of an overworked, grumpy sysadmin who's personal view of his priorities are decoupled from the larger organization's mission (usually taking care of customers, selling products, operating efficiently, and so on).

In this particular example, what is important is that the new customer whats to talk with someone about solving his problems. That someone is me, and I want to impress him, sell him something that will truely meet his needs, and hopefully turn him from "new customer" into "repeat customer" or even "loyal customer". THAT is what is important, and getting the customer's file quickly and easily with minimal hassle is merely a tool that enables the truely important work to happen.

Not having the email for 1 hour means I'll either have to call him back in an hour, while he probably calls some competitors and shops around. Often times people will buy from the first friendly, knowledgable person who goes to some effort to help them.... searching until they find that person/company. Delaying response to a new customer by 1 hours would put me at a competitive disadvantage.

Or we'll have to proceed without it (FTP is not an option), leading to frustration as he explains material that would have been much better delivered as a file. Maybe it would go ok, maybe not. But it's starting the whole process "on the wrong foot".

Then again, if your business is being a grumpy sysadmin where you have (captive) "users" rather than "customers", maybe delaying new email conversations is a big advantage which is not offset by any impact in "responsiveness" because it's already intentionally low.

One good point about this proposal

It deals with spam at the server level. All the wonderful user-level solutions don't do jack to stop spam from being sent. Look at the numbers the spammers show for return rate, and look at how fast spam programs can go, and you'll see that the only solutions that will work are those that make it expensive to send spam. Anything else will just make the spammers send more spam to try and get the hit rate they need.

The real solution

We should grab some of the guys who get 1000+ spams per day, point them to the physical location of the spammers, and then step back. I can guarantee you that vigilante justice is entirely appropriate here, considering we want the gov to step back from the 'net instead of entering new "secret arrests of spammers"(?) laws.

Bogofilter does pretty well for a client filter

The summary does not seem completely accurate; since the greylisting MTA sends an SMTP temp failure there should never be any false positives as long as the sending MTA is vaguely RFC-compliant (sadly not true I suspect). Or at least that was my reading of the paper...

I'm currently using Bogofilter [sourceforge.net] (and looking into CRM114 [sourceforge.net]) and getting better than 99% accuracy (about 1 in 200 false negatives at the moment) and very very few false positives (maybe 2 in 5000 messages).

Of course these are MUA level filters (and yes, I know, I've already "paid" with bandwidth to download the spam) - however since the proposed "greylister" would have to be installed as the MTA at major ISPs (as the authors note) I'm not convinced that is more likely to get widespread adoption than the various sorts of adaptive client-based filtering now available, particularly as it requires a database to back the method up.

As far as I am concerned the major factor in a spam filter should be zero false positives - personally I don't mind reviewing one or two spams a week but I get really annoyed if I were to lose a real message (note the two false positives I have sent to date with bogofilter contained forwarded sales pitches along with a message).

co-evolution

During the initial testing of Greylisting, it was observed that the vast majority of spam appears to be sent from applications designed specifically for spamming. These applications appear to adopt the "fire-and-forget" methodology.

Spam guards and spam co-evolve. Since greylisting is easy to get around by spammers, if it becomes widespread, spammers will take measures to avoid it, and the net result will be a lot of extra traffic.

In fact, the impact of this kind of system on mail could be pretty bad if widely adopted: large amounts of mail may end up being held up in delivering servers, and "informative" messages sent by helpful mail systems (about "temporary failures") may end up creating more junk mail than they avoid.

97%? not impressive. It's POPfile for me

I get 98-98.5% accuracy with POPfile [sourceforge.net]. I get about 200 mails a day, of which around 30% spam. I get about 1 false negative a day, and maybe 2 or 3 false positives a month. It's a personal solution and as such is much more attractive to me than something server-based which has to be installed by a [typically VERY uncooperative] BOFH.

I use it experimentally for general mail classification (business/personal/a variety of mailing lists etc., all in all 7 buckets) on my home machine, and it works fine in these conditions too, although the accuracy is a bit lower (around 95%).

missed the point

I've seen some comments that say (paraphrasing) "For real SPAM filtering use <POPFile|Spamassassin|...>", but these missed the point (or perhaps didn't read the paper?). This method is a "first-pass" filter, getting rid of e-mails for which no redelivery attempt will be made. The second-pass filter should still be implemented for everything that gets through the first pass. From the paper:

"The Greylisting method proposed in this paper is a complimentary method to other existing and yet-to-be-designed spam control systems, and is not intended as a replacement for those other methods. In fact, it is expected that spammers will eventually try to minimise the effectiveness of this method of blocking, and Greylisting is designed to limit options available to the spammer when attempting to do so."

EOL SMTP

Or we could all abandon SMTP and move to my jabber-based email "solution".

What? Where is it? Oh, I'm still working on it. You can send and receive, but buddy lists are not implemented yet.

How about DSPAM ?

I quite like the idea of this greylisting, but it seems a lot of spam is nowadays being sent as DSPAM (cf DDOS) or Distributed Spam. A spammer infects a load of broadband machines with a simple trojan, and then calls upon a number of the trojans to send an email spam via that machines normal MTA (ie for most windows machines it uploads to the ISPs mail servers).

I know this is happening as some complete bastard seems to be doing this using my domain as a "From:" address (well, [random-word]@schmerg.com), meaning that I'm been getting about 30 or 40 bounce messages a day for the last 2 or 3 months now. And although the odd sending IP is repeated, mostly they're all from different IP addresses. And of course I'm getting perfectly valid looking bounce messages from perfectly reasonable companies (and only a couple of abusive replies so far).

Now the problem is that the email is being uploaded to thier (non-open-relay) ISP's mailserver that will retry properly, and anyone else looking at the IP address will see a perfectly reasonable IP (the spammer seems to gave infected a lot of AT+T customers, ComCast customers, etc.). So short of blocking spam on subject, this spam is harder to prevent in the first place.

I've semi-automated a process to report the infected machines (that provoked a bounce message) to the appropriate ISP, and seem to havign some success in getting the ISPs to contact their customers, but I think this new form of spamming using a distributed attack will be particularly hard to block.

Anyone with a great idea (or who knows more about this scheme, or the identity of the twat behind it) I'd love to hear from you...

--
T

SpamAssassin

The comments in this paper about other systems ignore one of the oldest and largest SPAM filters: SpamAssassin.

SpamAssassin can also be used at the MTA-level, and while this tool might be an interesting test to integrate with SA, its claims that other systems cannot feed back to the sender that their mail has been blocked is flat-out wrong.

Most people do not do this because you are almost certainly getting this mail through a relay, and that relay is going to get the SMTP temporary error and try to send a warning to the user who sent it. Spammers regularly slam my home mail server by using my address as the "From" in an entire batch of spam. It's pretty seriously annoying to get that deluge of junk, and it's not really necessary. If your spam system just identifies spam and lets the user (or sysadmin) decide how to deal with it based on how "spamish" it is, you get a much more reasonable behavior.

I junk thousands of pieces of spam every week, and I *never* junk valid mail. Yes, I do have some spam in my inbox. Most of it is tagged as potential spam, and I delete that after cursory inspection of the from addresses. Some of it is missed, and the overhead that I suffer having to identify that myself is amazingly low compared to not being able to read my mail prior to SA.

Check out SA. The latest version is pretty impressive, and if this "new" technique (I don't think the idea of tracking connection quality is very new, it's certainly done in SA to some extent) turns out to be useful... well SA works on much the same principal as Perl: There's More Than One Way To Do It. Bayes, Blacklists, Whitelists, Obfuscation detection, Checksum trackers, you name it, SA uses it. None of these techniques gets to say "this is spam", they all just get to poke a message in the direction of being spam or non-spam. This leads to something far more reliable than any one techniqe.

HOW SPAMMERS WILL DEFEAT THIS:

I really like the idea, and I think it will work wonders (if you are willing to accept your nearly-instant email now having approx. 1 hour delay).

However, here is how the spammers will adapt:
MailBlast 2.0 will send each mail TWICE. The first time to get the triplet on the greylist, the second time to actually send it. Or a little more sophisticated, it will run in one-hour blocks, and at the end of the hour, re-run the previous hours emails. No queue or other real-SMTP server functionality necessary.

Wild idea, maybe ISP' should do something at

the sender level.

Like, say, putting a maximum limit on the number and size of e-mails that can be sent out a day.

Gather studies on how many people send 1 to n e-mails a day, and how many people send out e-mails of 1 byte to n bytes in size.

My guess is, it's a pretty distorted curve, with maybe a few thousand people -- of all those online -- sending out millions of e-mails a day. The maximum most "normal" people will send a day is probably 100 (and that's a large over-estimate).

My Anti-Spam Idea

Here are my latest thoughts [intercarve.net] on winning the spam war.

I've submitted it to Slashdot. They rejected it. Tell me what you think. I'd like reactive approaches to get discussed a bit more. If you do too, submit this to Slashdot :]

Anti-Spam Techniques: Honeypot spam detection!

Aside from the obvious of getting the authorities to crack down on the existing illegal activities (relay hijacking, violation of TOS of ISPs, header forging, etc.) which is the only true solution, I think there are much better approaches than this "greylisting" method.

The problem with the greylist method is it still slows down mail service, and potentially more than the relay blacklist features. The objective here is that end-user/networks should not be penalized in the fight against spam. We already waste too many resources, and according to my latest mail server stats, more than 65% of our inbound mail is UCE. I'm fed up with more than half my e-mail bandwidth being crap my users didn't request so more resource allocation on a local level in the fight against spam is counterproductive!

Here's a very clever, much more practical method I cound recently.

A company is Canada has set up what it calls SORBS [sorbs.net]: Spam and Open Relay Blocking System.

What's different from their blacklist is that they maintain "honeypots" strategically located around the Internet. These are servers they specifically set up as inbound mail relays, but never for legitimate purposes. If the servers get [select] mail activity, it's assumed to not be legitimate and it flags the source as a potential spammer... it makes a lot of sense. You create a domain name, but don't promote it in any legitimate manner, and/or you seed spam lists with these e-mail addresses and then let the spammers send to your key systems around the internet and *bam*, they're identified in real time, and then added to a blacklist.

I really like this idea. Like any other system, it has the potential for abuse but the beauty is the identity of the honeypot systems is kept secret, so it's very difficult for anyone other than spammers to exploit the network.

Re:Sarxpam

find the people who sent it, and send them a message saying "I'm Ron Jeremy... I don't think you *want* me to have another 3 inches"