EarthLink Is Losing a Lot of Email

LandGator writes "Robert X. Cringely, doyen compu-columnist for PBS, reports on a hidden e-mail problem at Earthlink: They're losing up to 9 messages out of 10, found as a result of a friend's testing." From the article: "He sent messages from other accounts to his Earthlink address, to his aliased Blackberry address, and to his Gmail account. For every 10 messages sent, 1-2 arrived in his Earthlink mailbox, 1-2 (not necessarily the SAME 1-2) on his Blackberry, and all 10 arrived with Gmail. Swimming upstream through Earthlink customer support, my buddy finally found a technical contact who freely acknowledged the problem. Since June, he was told, Earthlink's mail system has been so overloaded that some users have been missing up to 90 percent of their incoming e-mail. It isn't bounced back to senders; it just disappears. And Earthlink hasn't mentioned the problem to these affected customers unless they complain."

DIY

[tverbeek]

This sort of thing is the reason I host my own e-mail. At least this way I usually know when it's broken, and I have the opportunity to fix it.

Lost e-mail? WHAT THE HECK?

[matth]

E-mail should never be lost! We had an issue where I work with e-mail BACKING UP for a few months while we implimented new mail servers... but no mail was ever lost.. it either got bounced back (not usually) or would arrive several hours after it was sent. To actually LOOSE e-mail indicates that Earthlink is ACCEPTING the mail and then DUMPING it!!!! When our servers were overloaded, we just rejected the connection, until the mail server could handle more mail.. and then we accepted it.

Re:Lost e-mail? WHAT THE HECK?

[the linux geek]

Exactly. At the very least it should be saving the email and sending it later, not just ignoring it. It is not that complicated; it's cetainly technically feasible. Of course, this is Earthlink we're talking about. I used to have these guys as my ISP before cable became widely available, and the system for connecting was a mess (If you tried to use Outlook instead of Outlook Express, for instance, it would require uninstalling/reinstalling their client software.)

Re:Lost e-mail? WHAT THE HECK?

[matth]

Actually we handle close to 50,000 e-mails per minute.... I happen to work for a largeish ISP in Pennsylvania :)

Re:DIY

[jandrese]

I'm surprised you can do this. Most ISPs block port 25 traffic that isn't sent to or from their own badly overloaded email servers. The more breathless among us might even claim that spam is outright killing email as a communication tool.

Re:Says a lot..

[morgan_greywolf]

Generally when you're stating statistics like "up to 9 out of 10", that means that in at least one your test runs, whatever it was you were testing hit 9 out of 10 times, in this case the fact that Earthlink lost e-mails. You say "up to 9 out 10," because in other runs it may have been 1 out of 10, 3 out of 10, or 7 out of 10. You're trying to show that how bad it gets. When stating the statistic for useful statistical purposes, however, one should definitely also give averages, like: "overall Earthlink lost 50%". Now, in this case, Cringely actually states that his friend tried several times to send messages a block of 10 messages to Earthlink account, to an aliased Blackberry account, and to his Gmail account. Each time, Only 1-2 made it to the Earthlink and to the Blackberry (aliased off the Earthlink account), and all 10 made it to Gmail.

Re:Says a lot..

[Southpaw018]

Parent is +4 insightful? Wtf? The quote specifically says the average was 8-9 lost. Parent is implying that maybe only 1 was lost for the most part, or something similar.

C'mon, can people not even be bothered to read the article SUMMARY any more?

No guaranteed email delivery

[martin]

Lets remember way back when this happened all the time shall we.....and we used to say to the users.....

"there is no guarantee of email delivery" (and optionally "Get over it")

Remember this folks, no where in the RFC's is there anything that states email will get delivered....

Just because all us sys-admins do such a great job, most of the time it does get there, people forget the dark ages of the internet when this would happen all the time.

OK 90% email loss is really really bad, and it use to be more like 5% loss (at worst), but people need to remember email isn't guaranteed.
 

Re:Lost e-mail? WHAT THE HECK?

[SecurityGuy]

Never? I think you have an unrealistic expectation of the quality of service you can expect from email. I don't believe email has ever guaranteed delivery. We've just gotten used to the fact that like the postal orifice, it nearly always works, so we act like it really always works.

That said, 90% failure is ridiculous. :)

Re:Says a lot..

[hey!]

Usually the phrase "up to" is used to make the data sound like it says more than it does. It literally means "possibly as much as but no more than", but people read it as "on average". Using "up to" to make a claim that sounds broader than it really is is, as you imply, dishonest.

In this case what they have is an estimate of the average. But it's not based on much data or systematic testing. By saying "up to" they are actually encouraging the reader to interpret their results narrowlly. They could have said "9 out of 10 times", but by saying "up to 9 out of ten", they imply (a) they have no evidence that the system performs worse than this and (b) the system may at times perform better than this.

Overall, I think this is an example of honesty.

Re:Wonder if they can be sued

[Thraxen]

Well.. let's think about this. They have had the problem since JUNE. That's a damn long time to be losing up to 90% of a customer's emails. On top of that, instead of notifying all their customers of the issue, they only tell them if they call to ask. So how is a customer supposed to know that some of their e-mail is missing? It's hard to know about something you never had. The only way you would likely figure it out is if e-mail you expected to receive were repeatedly not making it through. Even then, the first couple of times you would likely figure that the person/company that was supposed to be sending the e-mail simply never did.

The example to which you responded is a perfectly valid concern and is an example of the type of e-mail a customer would never know is missing. They really deserve some sort of litigation. A simple notice sent out to all your customers back in June would have allowed their customers to switch any important accounts tied to that address before much was lost. But now, SIX MONTHS LATER, a lot of people have likely missed out on a lot of important messages.

Re:Lost e-mail? WHAT THE HECK?

[scatter_gather]

The extra "o" is for emphasis. The more "o"s, the loster it is. Glad I could clear that up for ya.

Re:No guaranteed email delivery

[soccerisgod]

Just because all us sys-admins do such a great job, most of the time it does get there, people forget the dark ages of the internet when this would happen all the time.

Back then, you may have had an excuse. Today, the excuse that the RFC doesn't specify email gets delivered should get you fired for being a failure who doesn't give a shit. Just my $0.02.

Re:Lost e-mail? WHAT THE HECK?

[silas_moeckel]

If your antispam and antivirus are running after you have accepted the email your system in broken. This is all companies playing fast and loose with the RFC's you should never accept mail that you are not going to deliver. This really is not that hard to run spam and virus checks as milters (or whatever your email application does) or place front ends that do so.

Re:DIY

[cayenne8]

"Technically that is against the ToS for regular Earthlink accounts."

So...get a business acct. I had one with Cox..was great. Static IP...no caps on uploads or downloads...could run all the servers I wanted, and a low level SLA. The one time I had trouble, I called the support line..rather than put me on hold, I left a msg. with my problem. In about 3 min..I had a tech call ME back.

It was only about $70/mo....I'd highly recommend it.

Bullshit

[ZombieSquirrel]

Something stinks here. This article does not have a lot to stand on. "A friend's testing"?! How scientific is that? And anyway, Earthlink is not exactly a fly-by-night operation. Don't you think more people would have noticed if 9 out of 10 of their emails were disapearing since June!? No way. This is crap. I have two earthlink accounts and I haven't noticed anything. Maybe his "friend" is just an idiot. Maybe Cringely is just an idiot I have nothing for or against Earthlink, I just hate bad information.

Re:Semantic Man!

[value_added]

Technically, we do not know that the email was *lost*, nor can we ever know.

Seeing a black sheep in the field does not prove that every sheep is black, nor that there is at least one black sheep. All you can prove is that there is at least one sheep that is black on at least one side.

Donald Rumsfeld, is that you?

As An Earthlink Customer

[mpapet]

I have no problems. Plenty of spam and the good stuff comes through for me and my wife.

Believe it or not, they've been very good. The one issue I've had they resolved quickly once I was escalated above first-level tech support.

Maybe it's a location-specific issue?

I can qualify another post that talked about the new email sender verification thing. I get it sending mail from the web email interface. But none of my friends or the emails I send myself from work require sender verification.

I don't know what the motivation behind these complaints may be. It certainly isn't bothering me or my wife. Maybe it will..

(shrugs)

Re:Semantic Man!

[DoorFrame]

There are known emails and unknown emails, and those divide into the known unknown emails and the unknown unknown emails. THOSE are the ones you've got to watch out for.

Re:As An Earthlink Customer

[N7DR]

Ditto and likewise. No problem here at all, and I would definitely notice it (and have noticed it in the past, within less than an hour of a problem occurring). So maybe Earthlink is losing a ton of e-mail on some accounts, but it seems to be working fine for others. I really can't believe that the problem can be widespread. People may be sheep, but surely if Earthlink were losing any noticeable fraction of most people's e-mail, they would have suffered a mass defection.

Re:Lost e-mail? WHAT THE HECK?

[Anonymous Coward]

It's a troll because mail wasn't delayed for a few months (as you and the retarded troll seem to think), but there were a few months where, during heavy times, mail was delayed for an hour or two.

When you move out of your parent's basement, you might realize that it is perfectly reasonable for the turn-up of a new mail service to take several months of planning, execution, and testing to ensure that no mail is actually lost when you are dealing with 50k messages an hour.

Re:DIY

[walt-sjc]

If you don't want shit service it's going to cost you. The old saying "you get what you pay for" applies. If you don't need to run servers, have a static IP, or a better TOS, then don't pay for them. If you do, then it's not free. Obviously the market supports the extra cost.

But back on topic here, earthlink has had email problems for YEARS. The only difference now is that the problem is worse than it was and more people are noticing.