The sad part is, I did hold back. Mostly, due to post length and the fact that I don't want to spend the next week writing it up.
Suffice it to say, that I have an archive of 100s of Rackspace emails, and 60 or 70 phone calls, all stored because we were positive we'd have to sue their ass.
Yes, they were that bad, and showed that much incompetence.
Not Rackspace.
There are lots of other fish out there.
Rackspace?!
Wait, what?!
Rackspace is the most *horribly* run hosting service of all time. I could go on for hours and hours and HOURS describing how inept and incapable they are.
From months to source SSDs, to providing horrible support, and utter incompetence on the part of their staff... I mean, they're HORRIBLE! Just plain horrible. If any of their automation breaks down? Well, good luck getting help FAST. I mean, if a VM move fails, well.. maybe you'll get help in 24 *hours*.
Maybe. If it's the weekend, well.. or at night... well, after all, people only use the internet during the day!
And if anything is even slightly outside of the box? Good luck with that!
No, no, no. Not to mention, expensive. I was saddled with these boneheads when a PHB decided they were a great idea! Meanwhile, they take MORE time out of your day, than just maintaining hardware servers in a data center, because if anything goes wrong?
Well, emails, calls, conferences, blah blah blah. In 1/10th of the time it would take for rackspace to fix ANYTHING, I could just tell a traditional data center to reboot my box, or install a new one.
Hell, I've had VMs@Rackspace that were HUNG, that would NOT respond to the web console reboot command. TIme to get that fixed? HOURS. Christ, just GET IT FIXED.
And cost? COST! PHB made me use these boneheads. We leased two Dell R720s. For the cost of 3 MONTHS worth of the lease, I could have bought a better equipped R720! Or, hey, maybe TWO Supermicro servers!
Rackspace is a time sucking hole in the ground. It's "expert" admins will suck your time away. Hell, I had to put off dozens of projects, whilst I dealt with their constant and continual fuckups, the phone calls, the emails, the explaining to them how to fix simple thing!
Heck, don't even get me started with Rackconnect, good god. Worse, buggy as hell as it is (or at least was), they had all sorts of problems with their automated iptables scripts. I snag it, debug it, and realise that some conehead there can't write simple bash...
Fix it...
Report the fix...
And am still suck with months, I repeat MONTHS of their script being used on my boxes, with no way to replace it (it was scp'd in on boot), and therefore broken firewall rules all over the place. MONTHS, when I provided them with a fix! A ONE LINE FIX AT THAT!
No, no, no, no, NO they are horrible, stay away, run the other way, my god stay the hell away from Rackspace, the most useless company on the planet!
If any of you, I repeat ANY of you want more detailed info, please let me know.... I hope they burn in flames as they go down into a tarpit in hell!
Don't hate me.
Trusting that your *only* full backup is good, isn't a great policy either. I tend to do full backups every quarter, but it depends upon the data set, and of course, the size of the data set. If the data set is trivial... then who cares? Do it weekly.
Yeah
A 24 hour old sync isn't a backup. It's a slightly delayed mirror.
"Rotational backups" isn't just a single thing. It's a whole ball of wax. Part of that ball of wax, are test restores. Another part of that are backups that only sync changes, something exceptionally easy with rotational backups, but not as was with a filesystem snapshot.
In 10 seconds, I can run 'find' on a set of rotational backups I have, that go back FIVE YEARS and find every instance of a single file that has changed on a daily basis. How does someone do that with ZFS snapshots? This is something that is key when debugging corrupt , or looking for a point to start a restore from (someone hacks in).
Not to mention that ZFS could be producing corrupt snapshots -- what an annoyance to have to constant restore those, then do tests on the entire snapshot to verify the data.
What I see here is a reluctance to do the right thing, and a desire to think that the way people do traditional backups is silly.
Git has no rotational backup ability in it. You can't do rotational backups of the machine, on the machine for starters!
ZFS is not a rotational backup as well!
Failure, 101, backups. Go back to school.
Both of the above solutions do not prevent slow corruption, and they do not prevent issues where the machine is suspect. (Yes, ZFS can have bugs). They also do not help if the machine has been hacked into. They don't help if there is a fire, flood, or theft of the local box.
Modern backup methodology has been developed over decades of people suffering JUST THROUGH THIS VERY THING. If you plan to just throw all that away, and pretend everyone doing backups is an idiot -- MAKE SURE YOU KNOW WHAT YOU ARE DOING.
Because -- this very issue would not have been even a tiny concern, if proper, off machine, rotational backups were being done. And, if you aren't going to follow proper backup methodology, then you'd better sit down in a quite place for a few hours, and think of every possible disaster scenario, AND issues with the code you're going to be using for those backups.
Hell, this whole KDE problem started, because the people using it did not even know how git works, 100%! Now, you're suggesting that using another tool, ON THE SAME BOX, is the answer? What will someone miss on ZFS?
No, please, think about this more carefully.
Good grief!
After all of that, not a single proposed solution is a proper, rotational backup.
This is what rotational backups are FOR. They let you go back months in time, and even do post-corruption, or post-cracking examination of the machine that went down!
Backups do *not* need to be done to tape, but a mirror or a raid card is NOT a backup. This is actually simple, simple stuff, and it seems like the admins at KDE are a bit wet behind the ears, in terms of backups.
They probably think that because backups used to mean tape, that's old tech, and no one does that.
Not so! Many organizations I admin, and many others I know of, simply do off-site rotational backups using rsync + rotation scripts. This is the key part, copies of the data as it changes over time. You *never* overwrite your backups, EVER.
And with proper rotational backups, only the changed data is backed up, so the daily backup size is not as large as you might think. I doubt the entire KDE git tree changes by even 0.1% every day.
Rotational backups -- works like a charm, would completely prevent any concern or issue with a problem like this, and IT IS WHAT YOU NEED TO BE DOING, ALWAYS!
I suggest it be made very clear what data is collected and precisely how it is used.
Then let people decide if they want to use the service.
Right now, the only choice is to GUESS how the data is being used, and to GUESS precisely what is being collected. That needs to change.
Outside of the above... Google behaves well? Pfft. They behave as poorly as any large corporation, from what I've seen. Further, as mentioned above, the sort of "if you don't like them, stop talking about it, just don't use them" thought process is broken.
Next you'll suggest that when I go to a garage, and they do something I don't like -- I should just be quiet about it/? Just switch, and not tell anyone about my experience? Yeah, that sort of reason is good for everyone, heck, it even breaks the free market!
People -- stop getting upset if someone says something about a company. It isn't a person, after all. Google's feelings won't be hurt. Why are you protecting them? Hell.. people aren't even saying "They don't collect data"... no, they just say "Don't use them then".
That's akin to standing up for your brother, after he stabbed someone. Except... Google isn't your brother, they aren't your friend. They don't care if you live or die. They just want to make as much profit off of you as possible.
What is the real goal here? To stand up for someone that does everything they can do, to reap every penny off of everything you do?
Ah, a new tact -- no one is forcing you to use Google, therefore it's OK that they do whatever they do.
No one is forcing you to rent a particular apartment either, so I guess it's OK if the landlord puts cameras in it, and spies on you?
No one is forcing you to go to a particular grocery store. I guess it is OK for that grocery store to poison your food, if you don't like it, shop elsewhere?
Sorry, the "if you don't like that you're being spied on, just shut the hell up and stop using that product" is another red herring. Please stop with the Google fanboism, OK?
Your response is the equivalent of stating that since Microsoft murdered someone, I shouldn't be upset that Google did. Further, since we all know Microsoft murdered someone, I am out-of-line for mentioning that Google did.
Guess what Jimmy -- lots of people mention the bad things that M$ does. My post is about the bad things Google does -- and they do LOTS of bad things.
And I call them on those bad things, and the bad things they continue to do.
Awesome... now more people will be tricked into switching to Google's DNS servers, and therefore, more people can be tracked by Google.
Before, Google just watched your browsing habits, your email, your phone calls and cell phone activities, your physical connection, tracked you through advertising, monitored your connections to your friends, and, well, when you took a dump too.
Now, Google plans to monitor every other activity your computer partakes in, as it watches all the DNS lookups you make. Any website you go to, that is not done via a Google search. What other software you use. What forums you go to. What *threads* you look at in forums, as the dns entries will sync with threads Google has already cached. Do you download torrents? Do a lot of MX record lookups?
Google can determine a vast amount of info via DNS lookups.
Google -- can you PLEASE just focus on making your core, search technology less inane? Not everyone wants to search for random, unrelated responses to searches. When they search for "bob cat", they don't want "Robert Kats".
Oh? And while you're at it, please make Verbatim searches work again. You've only had that for what, a year since you SCREWED UP + SEARCHES, and you've already started to DEGRADE IT!
Cornholes!
Yes, yes, yes, OH GOD YES!
I've been wanting to eat one of these birds for DECADES! A bird so tasty, we hunted it to extinction!
There are recipes I wanted to try! Pies and stews and just cooked in the oven. They should do a kickstarter, I'd kick in!
Users who replaced their TP-Link firmware with Open/DD-WRT firmware can sleep well.
On what basis?
On the basis of the security updates that occur, every single time a kernel or userland vulnerability is discovered?
I have yet to see a security release for DD-WRT. I see updates, randomly, which have nothing to do with security issues. Certainly the stable branches of both projects releases rarely.
Note, I'm not faulting these guys -- these are nice firmwares. However, to think that they are somehow more secure, when they fall prey to the same problem -- THAT IS, NO UPDATES DUE TO SECURITY ISSUES, is strange. These firmwares are just as insecure as stock firmwares.... in fact, likely MORE insecure, because what stock firmware has SSH exposed? Or other userland tools exposed?
Stock firmwares seldom have fluff modz, whereas one of the strengths of DD-WRT and others is to expose these services on user request.
If I look at
Where were the automatic, day-of-announcement updates for DD-WRT? OpenWRT? Until these tools incorporate Debian (or other copycat distros) updates with tools like apt-get, these things are WORSE than standard router firmwares.
What we need is to standardize these router firmwares on something like Debian.
Get hold of portable property. -- Charles Dickens, "Great Expectations"
