Phone conversation between two data center techs:

Tech 1: "There's a cloud in the Facebook datacenter!"

Tech 2: "So? Facebook is built on cloud technology!"

Tech 1: "No I mean a real cloud!"

Tech 2: "Facebook is built on a server cloud architecture. It IS a real cloud you idiot!"

Tech 1: "There is a real cloud with real rain in the data center you geek retard! Its shutting down the servers!"

Tech 2: "Servers shutting down? Maybe the rainfall service is flooding the network with raindrop packets? That would be an emergency! Facebook's cloud is overloaded!"

Tech 1: "WTF are you talking about? There's real water in the servers. The water is causing electrical faults. Humidity is really high."

Tech 2: "There's a humidity problem? Probably the flood of raindrop packets interfered with the environmental control service.

Tech 1: "You might be on to something. Maybe that's what caused the cloud in the first place?"

Tech 2: "Caused the cloud? The environmental controls have nothing to do with the Facebook cloud!"

Tech 1: "You idiot! There is a real meteorological cloud in the data center, complete with real rain! Everything is getting wet! I'm not talking about the Facebook cloud!"

Tech 2: "So what's the problem? Just adjust the environmental controls and reduce the humidity. At least the Facebook's cloud and raindrop service are ok. You really had me worried about Facebook's cloud for a second."

Tech 1: "The entire data center is powering off you flaming moron!"

Tech 2: "Wow that's a real emergency! Facebook can't operate without its cloud!"

Tech 1: "Right. So back to the real cloud in the data center. I'm looking at it right now. Its unreal! You should see it!"

Tech 2: "Great! Now that the power is back I'll VPN in and check on the cloud. Are you seeing a lot of raindrop packets on the network?"

You're spot-on about Google Maps. I've never seen a better example of an app solving anti-problems and a few real problems but earning little revenue for it, when it could actually solve big and financially valuable problems if Google chose to.

I'd love to ask Larry Page and Sergey Brin: "How are you content to spend $1.5 billion a year on research yet $12.9 out of $14.4 billion of your 2012 revenue is from showing ads on search results?" $1.5 billion a year can solve a lot of big and valuable problems based on what my software development team can do on a few million a year.

Disagree on many points:

Guaranteed minimum income is a socialistic recipe for disaster. Take away all financial incentive to providing value or solving valuable problems and people will just sit on their asses, eat and watch porn all day long.

The "structural malaise" is caused by years of poor education in the US, which leads to poor motivation, depression, bad choices, etc. Why do you think US employers push so hard for H1B visas? I receive a couple resumes a day for positions at my employer, a major tech company. The average skill set, experience and motivation in "tech" workers is abysmal today. People padding their resumes right and left with skills they were only briefly exposed to and didn't actually learn or use. Resumes and interviewees that naively don't convey the value they bring to an employer. My favorite interview questions are "as an engineer... what problems in the world have you thought about solving?" or "do you have any software engineering hobbies?". Yet I get so many blank stares in response to those questions. It can be scary when we actually draw an algorithm problem on a white board and merely ask them to try and think of a solution.

I work in government research. Its not drying up. Its expanding by leaps and bounds. Sequestration has been a good thing. Its encouraged Govt agencies and contractors to trim their fat. Large prime contractors with high rates have lost their contracts and their top talent to contractors with lower rates. Its survival of the fittest / adaptation towards a better future.

There is no human tragedy coming. The employment landscape is changing. We rode high and affluent in the last 20 years on the backs of low interest rates and trillions of investment dollars combined with dirt cheap labor in Asia resulting in unusually high growth and economic expansion. Money and jobs were everywhere. This is the correction to more normal times. Biggest examples are people driving smaller and more efficient cars and manufacturing moving back to the USA now that China's workforce is smarter and can't be exploited like it once did. Did you hear about Motorola hiring 2000 workers in Texas to manufacture the first ever US-built smartphone?

The sooner people forget about the last 20 years and adapt to the current world the better everything will be.

I have pursued both anti-problems and big problems.

"Anti-problems" are everywhere and discussed frequently because the public and press are familiar with the problems, solutions and rewards. Everyone knows the possible financial reward from Facebook/Yahoo buying your social networking website/app, developing the next great web technology at Google, or developing financial software on Wall Street. These "anti-problems" are very sexy too. Investors buy into the hype. I used to work in finance. I saw investors bringing $50 million investment utterly fail in their due diligence and believe total lies. So anti-problems can attract a ton of easy investment... if conning investors is your thing.

But anti-problems bring deep risks, which you don't generally hear about. I've seen great software engineers work for very low pay because their employer or partners are trapping, baiting and exploiting them with equity promises, shared ownership obligations, using financial pain and desire to own their IP as motivation. I have seen careers ruined due to unseen risks that only good (and expensive) legal counsel would have helped you avoid in advance. But you rarely hear about these risks because they're just not sexy enough to print or are deeply confidential.

I've developed solutions to anti-problems that generated millions in revenue. I'm not a millionaire today because of simple greed. I did not secure ownership in advance and faced a costly litigation to regain my ownership or fair compensation from a very large company. I learned from these experiences. I learned to walk away from lead engineering employment positions where I was planning to generate or advance IP worth millions or billions and my employer refused to compensate me fairly based on the value of the IP.

Nor do you hear about "big problems". Sometimes its because fewer understand the limits of physics. Fewer understand the deep math and technology expertise involved. But mostly its a matter of the general public and press not knowing a solution was even possible. Best example was the original iPhone which offered the first effective touchsceen on a cell phone. That simply wasn't on anyone's radar.

I work on one of these big problems. I am a technical lead on software that is currently saving the lives of US soldiers and improving their lethality... right now as we speak. Nearly every solider we demo our software to says to us "I need that NOW". We truly feel like Steve Jobs demonstrating the first iPhone. That's part of the reward when you pursue solutions to big problems. Yes we're also compensated very well.

Yet we struggle to recruit top engineering talent away from all the "anti-problems". I've been pursuing a few local software engineers for a few years. One is developing a social networking app and probably earning a small fraction of what we pay. I get the feeling certain engineers have pursued anti-problems for so long, they're afraid to abandon their hopes of a big buyout, switch to working on big problems and have to admit to themselves that pursuing anti-problems was a bad choice.

You are exaggerating.

First of all, US government solicitations can be vague or specific. When they are vague, it is intentional in order to encourage a wide variety of proposals. Have a look at: http://www.sbir.gov/solicitations

Now here is where you are very wrong. Bids and proposals are anything but "a very vague description of a project and associated budget". Maybe years ago, in some areas of the US Government this was true. Maybe its still true in a handful of areas. But right the majority of DoD proposals are very specific. I've composed, won and lost SBIR proposals. Vague SBIR proposals are rarely awarded.

First what not to do:

Don't invest in RAID-5 or RAID-6 arrays unless you can afford a high quality controller and SAS drives. Cheaper RAID arrays often have significant problems rebuilding. They're also expensive and large. You're also gambling that you choose a hard drive model that doesn't have any serious systemic problems. Choose poorly and you're risking total data loss. Besides, they key desired feature here is mirroring and one doesn't need RAID just for mirroring.

Forget about recordable DVDs or BluRay discs. The media capacity is small and thus cumbersome to mange. The recording dyes can degrade surprisingly fast, again varying greatly with manufacturer or lot. There's lots of research on this and you can download software to measure correctable errors before the discs become unreadable. I've had some discs become unreadable within 6 months. Some discs test with significant error rates immediately after being burned, before even leaving the drive tray. Basically if you're not testing error rates of your dye-based media, the joke's on you for buying them.

Tape drives are inconvenient for random access. Longevity could be better or worse than hard drives. Without hard data one can't recommend tape drives. Like everything else, longevity probably varies quite a bit between media brands and drives/recorders.
Internet backup services aren't going to scale depending on how much you need to access.

Here is what I do to manage about 3 TB of family photos and video, including about 0.5 TB of encrypted (truecrypt) content. Its not a perfect solution. However its cheap and easy to set up and maintain.

1) I have several "master" directory trees on my main computer. It runs Linux currently, but windows could work too. There are several drives and partitions, all mounted in a master "media" directory tree. There are subdirectories organized by year for which the media was generated. This helps with locating media of interest.

2) I have a large number of 1 to 2 TB external USB 2.0 hard drives from different manufactures (mostly Seagate and WD branded external drives from Sam's or Costco), each one clearly labeled. Some have one partition. Some have two partitions with the second partition being a truecrypt partition. The external drives are grouped together such that each group is a complete backup.

3) For each drive I have a unique shell script that calls the 'rsync' command.

4) I keep one group of drives at my house (the local backup), a second group in a bank safety deposit box and a third group at a family member's house. So that's the original plus three backups.

5) I sync the local backup about once or twice per week, or after significant media generation. This only takes a few minutes and is fully automated.

6) Once every month or two I will do a local sync, swap the local with one of the other groups then do another sync.

7) I periodically reformat the drives and check the SMART data.