UK ISP PlusNet Accidentally Deletes 700GB of Email 282
steste writes "A tale of email woe for PlusNET ISP. According to this announcement they have spent the last month attempting to recover 700GB of accidentally deleted emails. By their estimates, up to 12GB of these had yet to be read by their recipients. Despite the efforts of a data recovery specialist, they have now given up on recovering any of the deleted data.
Well that's one way to deal with spam." Spam is one thing; I just wonder how inevitable losses like this one square with the EU-wide data retention laws.
Bad news for those who use email as a file cabinet (Score:5, Interesting)
But I know people who use email accounts as a repository for their online lives. Gmail is encouraging this attitude, of course. Now I think Google is probably a little more responsible, but it does give one pause.
Now, for that unread email, that just sucks eggs for those poor people...
Data Recovery Specialist (Score:3, Interesting)
Re:And on and on and on... (Score:4, Interesting)
So...where were the backups? (Score:3, Interesting)
Or am I missing the point and actually 700gb of mail was just one day?
I'm a PlusNet user - but not for long (Score:5, Interesting)
I've been with PlusNet for years, and they were a pretty good ISP until a few months ago. Since then, we've had a string of problems, of which this is only the latest.
I've had my broadband connection out more than on for weeks at a time, for a start. This in itself is inexcusable. What's even more inexcusable is telling me I had to accept a significant penalty charge if they escalated the fault to my telephone service provider (BT) and they found no fault -- which doesn't sound unreasonable, until you know that the fault was evident using nothing but PlusNet-supplied hardware plugged into a BT-installed phone socket, with no complications whatsoever, and that PlusNet had already indicated that they themselves couldn't diagnose a fault. This was a total loss of service for hours at a time, several days a week, remember.
On top of that, they decided to forcibly upgrade everyone to "up to 8MB" broadband recently. The ethics of using that term are dubious at best: it's only for downloading; the highest recorded speeds off-peak are more like 5-6; and at peak times you'll be lucky to get more than 1-2. Moreover, they acknowledged ahead of time that there would be significant disruption (for weeks, not hours) to each customer after the upgrade, they said they wouldn't confirm when any given customer was being upgraded (so no idea whether the problems I had were to do with this or some more general issue, then) and they said some customers' performance would actually drop but they wouldn't revert the change if this happened. They had so many problems with this that they have now suspended/abandoned the process, and sent a grovelling e-mail message to their customers.
Their tech support people have also been completely over-run, partly due to inadequate resources and partly due to their own incompetence (e.g., they totally failed to read a note I'd helpfully left on their system for them clarifying a question they always ask, and asked the question in boilerplate form anyway). To add insult to injury, they've changed their phone system in ways that have repeatedly broken, and now mean you go through several layers of automated menus before talking to a real person. Yes, they really did tell me at one stage that if I was experiencing broadband connectivity faults, I could find more information on their web site.
And now, of course, we have the e-mail fiasco. It's not the first big e-mail problem: I've recently had legitimate and important messages from the sysadmins of another service I use being bounced because they "contained a virus". (Not according to the other service, whose admins I know and trust, nor according to one well-respected intermediate service that was involved in forwarding the mail.) Moreover, this occurred even when I disabled virus checking for incoming e-mail; they were blocking incoming messages to me against my explicit instructions. Oh, and their new webmail system is poor in functionality and so bug-ridden that you can actually lose data. Some of this, in particular an arbitrary time-out for composing mails using webmail, was regarded as a feature when I asked the support staff about it!
I don't know what's happened to PlusNet. Perhaps they have simply been victims of their own success, after getting very positive comments for years (they were widely regarded as one of the best ISPs in the UK for a while) and a consequent boost in custom? In any case, the mighty have well and truly fallen, and I (along with many other people I know) am currently investigating alternatives as a matter of urgency.
Re:Data Recovery Specialist (Score:3, Interesting)
700GB was definitely over a couple of disks, probably a RAID box. If you lose a RAID controller in SOME(not most or all) brands of RAID boxes, it's very difficult to rebuild the map. I saw an NStor RAID box lose a controller, and it was impossible to recover.
I wonder of how much was recovered was from tape? 12GB of unread email probably means "we didn't have a backup of the 12GB", though that's a LOT of email to be missing.
These Things Happen. (Score:3, Interesting)
It just goes to show, for PlusNet's customers, that electronic systems cannot be fully trusted, even when and if multiple instances of the data exist. We can approach an approximation of 100% reliability, but we can't ever fully eliminate the possibility of data loss, especially when human error is involved.
Another time in an incident that is mostly unrelated to the topic at hand but makes for a good story, I had a customer who lost their array in a PC server. The machine had an array of full height HDDs that would get so hot within five minutes of power up that you couldn't touch them without gloves. To top it off, the tape drive mechanism that was supposedly backing up the system was sitting directly above the arrary, with a backup job that had been running over and over nightly on the same cartridge for over two years (so you can be certain the tape was useless).
It took about three weeks, but I was eventually able to recover all the data on the array (so far as we were able to determine at the time).
Re:I guess if I look at my email (Score:5, Interesting)
Re:I'm a PlusNet user - but not for long (Score:4, Interesting)
Their customer service has never been particularly superb, it's true: they messed me around a bit when I was initially getting broadband installed, for example. Still, until maybe the start of this year, the tech support guys always seemed to know their stuff, and any problems I did have usually resulted in a fairly quick solution after a fairly short wait to speak to a real and knowledgable person on the phone. I've noticed a really obvious drop in their service levels over the past six months or so, though.
Re:Bad news for those who use email as a file cabi (Score:4, Interesting)
I have every e-mail I have ever sent or received except for spam. I can't count how many times this has been useful. I don't want to waste time trying to figure out what I should keep or not, I keep it all. I do keep all of them local on my own hardware though. This allows me to protect and backup my own data.
This is just a case of a poor backup strategy causing data loss that should not have happened.
Re:That's not what the laws are for (Score:4, Interesting)
Re:I guess if I look at my email (Score:5, Interesting)
I may be out of date on this... (Score:2, Interesting)
Re:I guess if I look at my email (Score:5, Interesting)
Then one day, I lost several hard drives in one go (storm). Last week backups were corrupted, had to roll back to 2 weeks backups. I lost 400 mails, my gf lost about 10 mails.
It's been six months, now I forward everything to several gmail/yahoo accounts, do a local backup of thunderbird's mboxes, and scp the backups to two different machines.
To this day, I still can hear the screams from my One And Deareast Female User when I cry myself to sleep.
Bottom line:
- If you are single, yeah, host your server, it's fun, you learn a lot of stuff.
- If you are not, paranoïa should be part of your base skillset.
Re:I guess if I look at my email (Score:4, Interesting)
Running your own mail server provides a number of advantages :
On the other hand, it does require a little knowhow (with the number of HOWTOs floating around it's really not that hard), a dedicated connexion (although nowadays with widespread xDSL or cable this is much less of a problem than it used to be just a few years ago) and some kind of backup system. Some ISPs won't let you run a SMTP server though, in which case there really isn't much you can do (especially if it's the only ISP in your area as seems to be frequently the case in the US).
For backups I use a little NAT box with 300GB of RAID 1 storage to dump the essential stuff on my network, including the mail directories. They are also copied to a secondary drive inside the mail server.
Of course as the story illustrates, there is no absolute security. Your house may burn down, an asteroid may obliterate your region, or you might just type a space in the wrong place in your "rm" command
Apart from that, I've run my own mail server for ages now and I know I wouldn't want it any other way.
Re:I guess if I look at my email (Score:4, Interesting)
Well, check around. I found most ISP's will offer a business acccount. It is a few dollars a month more than regular, but, like with mine from Cox Cable...I have:
- Static IP
- No bandwidth limits
- No Ports blocked
- Basic SLA
It isn't that much more money, and I can run any servers I want, hell, even make a little money hosting small site for people too...Check into the 'business account' that many if not most ISP's provide. It may cost a little more, but, that freedom to do and host what you please is quite worth it...to me at least.
Re:Crediting Users?!?!? (Score:2, Interesting)
Plusnet also accidentally deleted all of our websites that they were hosting on several occasions last year, without having any backups. They say that it is our duty to ensure that our websites are backed up properly.