You're not asking the right questions:

The first correct question is why on earth would someone need to access half a petabyte? In most cases the commonly accessed data is less than 1%. That's the amount of data that realistically needs to reside on disk. It never is more than 10% on such a large dataset. Everything else would be better placed on tape. Tiered storage is the answer to the first question. You have RAM, solid/flash storage (PCI based), fast disks, slow high capacity disks and tape. Choose your tiering wisely.

The second question you need to ask is how the customer needs to access that large datastore. In most cases you need serious metadata in parallel with that data. For Petabytes of data you cannot in most cases just use an intelligent tree structure. You need a web-site or an app to search that data and get the required "blob". For such an app you need a large database since you have 5M objects with searchable metadata (at 200MB/blob).

The third question is why do you have SAN as a premise? Do you want to put a clustered filesystem with 5-10 nodes? Probably Isilon or Oracle ZS3-2/ZS4-4 are your answer.

Fourth question: what are the requirements? (How many simultaneous clients? IOPS? Bandwidth? ACL support? Auditing? AD integration? Performance tuning?)

Fifth question: There is no such thing as 100% availability. The term disaster in Disaster Recovery is correctly placed. Set reasonable SLA expectations. If you go for five-nine availability it will triple the cost of the project. Keep in mind that synchronous replication is distance limited. Typically, for a small performance cost, the radius is 150 miles and everything above impacts a lot.

Even if you solve the problems above, if you want to share it via NFS/CIFS or something else you're going to run into troubles. Since CIFS was not realistically designed for clustered operation regardless of the distributed FS underneath the CIFS server, you get locking issues. Windows Explorer is a good example since it creates thumbs.db files, leaves them open and when you want to delete the folder you cannot unless you magically ask the same node that was serving you when it created the Thumbs.DB file. Apparently, the POSIX lock is transferred to the other server and stops you from deleting, but when Windows Explorer asks the other node who has the lock on the file you get screwed since the other server doesn't know. Posix locks are different from Windows locks. It affects all Likewise based products from EMC (VNX filler, Isilon, etc.) and it also affects the CIFS product from NetApp. I'm not sure about Samba CTDB though.
I would design a storage based on ZFS for the main tiers, exported via NFSv4 to the front-end nodes and have QFS on top of the whole thing in order to push rarely accessed data to Tape. The fronted nodes would be accessed via WebDAV by a portal in which you can also query the metadata with a serious DB behind it.

I've installed Isilon storage for 6000 xendesktop clients that all log-on at 9AM, i've worked on an SL8500, Exadata, various NetApp and Sun storages and I can tell you that you need to do a study. Have simulations with commodity hardware on smaller datasets to figure out the performance requirements and optimal access method (NAS, Web, etc.). Extrapolate the numbers, double them and ask for POC and demos from vendors, be it IBM, EMC, Oracle, NetApp or HP. Make sure that in the future, when you'll need 2PB you can expand in an affordable manner. Take care since vendors like IBM tend to use the least upgradable solution. They will do a demo with something that can hold 0,6PB in their max configuration and if you'll need to go larger you'll need a brand new solution from another vendor.

It's not worth doing it yourself since it will be time-consuming (at least 500 man-hours until production) and with at least 1 full-time employees for the storage. But if you must, look at Nexenta and the hardware that they recommend.

And remember to test DR failover scenarios.

Good luck!

I live in Romania, which has a similar situation to Bulgaria economically, so I know one or two things about average net wage, so fuck off.

If I remember correctly, it was the 3rd party auditors that made the economical recommendations that led Greece to bankruptcy. In a perfect world, the financial institutions and auditors that pushed Greece onto such a road would pay for the economical disaster that they directly contributed to. But I guess that they're busy giving bonuses to C*Os. If your financial consultant (or tax consultant) makes wrong calculations/projections/recommendations for you and puts you into default, wouldn't you seek compensation from him? You did pay him to give you realistic results. How can one country's rating go down from AAA to Junk in one day?

You're claiming that it's not fair, but the IMF and ECB gave Greece loans at rates that are not sustainable. I can get an EURO credit at a lower rate than Greece has. Furthermore, for Germany it's win/win. They bought out a lot of Greek companies for pennies. Think of OTE that was bought by Deutsche Telekom. I personally feel like this is looting and not helping out. Private corporations from the US, UK and Germany (financial and audit) bankrupted Greece with bad advice, while earning serious money for it (think Deloitte, S&P, etc.). When the bubble burst, the Greek government received help at ridiculously high rates from a few countries and multi-national institutions. Then came the major companies from those countries and bought everything for pennies. Afterwards, they are still complaining that the Greek can't make the payments.

I see your problem right there: you're reading it from Economist or Bloomberg. How about checking out the bare survival conditions of a lot of Greek citizens? Should Greece abandon them because Germany said austerity is the way? The Greek government's responsibility is to it's citizens. P.S.: I'm not Greek or German either. I don't live in Greece or Germany, but I try to get my news from newspapers that aren't necessarily in New York, London, Frankfurt, Tokyo or Hong Kong.

This reminds me of a classical line: "there is a world market for maybe five computers"

• Surveillance & Patrolling (public, intelligence services, military and private)
• Transport (think of those Pizza delivery SLAs going down to 10 minutes or Amazon)
• Leisure
• Emergency Services Support (Search & Rescue, Disaster Evaluation & Reconnaissance, etc.)
• Gardening, Crop Spray-ing, etc.
• IT Infrastructure Support for massive events (think wireless at a concert)
• Scientific measurements for weather/pollution/etc.

Would you please point to a study that states that fracking wells have a higher incidence of water contamination than normal classic oil or gas wells? Traditionally Romanian gas exploration has used hydraulic fracturing. The only difference is that we are now drilling deeper, as well as horizontally and we can exploit more from an existing deposit. To give you an idea: right now, out of all the electricity produced in Romania, only 39% is carbon producing (coal, heating oil, gas), the rest is non carbon producing (hydro, wind, nuclear, photovoltaic, biomass). You can see the real-time information on http://www.transelectrica.ro/w... . You an also see historical values http://www.transelectrica.ro/w... .
Romania has gone through a complete overhaul of it's energy sources in the past 20 years. We have an installed capacity of 23GW with a power usage between 4GW (low point in summer) and 9GW (max point in winter). In the past 10 years we've added 2,5GW of wind turbines (completely absent until then), and 1GW of photovoltaic. Since we still need gas (for now) and have ample reserves, why should we import from our "old adversary" instead of using our own?

I remember seeing at some point numbers. It didn't impress in a single thread, but could easily saturate a 10Gb link in multi-threaded tests. They tested an FTP server on a T2plus. Regarding cores, we have anything from dual UltraSPARC IIIi to T4 based systems including some M-class. I believe the T3-4 has the highest number of cores. It should be 64 cores and 512 threads, but a single Solaris instance can only see 256. I believe that the M9000 and M9000-64 should have the same problem, but the biggest M series I've worked with is M8000.

While I'm a Solaris admin for some time, I can tell you that it's not the best TCP/IP stack. It does have all the bells and whistles, but it's not even close to the speed of FreeBSD. It's actually not even in the same ballpark as FreeBSD. It's probably Linux fast if you tune it properly. It does have cool configuration, virtual switches, link aggregations, hardware crypto that can be usable by OpenSSL, OpenSSH, and ipsec but it's not even close speed-wise. But the cost of all those features basically means that it has mediocre performance for simple, yet performance-hungry scenarios.

You've got it. What I don't understand is why nuclear electricity is put in the same basket as coal and gas plants. The incidents that Nuclear has gone through in the past 60 years only reinforce my view that it's a safe solution. If given all the fsck-ups that gave us Chernobyl, Fukushima and 3 Mile Island that's all that happened I think that it's pretty much OK. I'm saying this because coal/thermal have their exhaust pipe problems which affect a much greater percent of the population and hydro is in general an ecological mess that also involves massive population relocation.

Nothing is laptop hardware in that machine. Like previous Mac Pros it has workstation cpu (Xeon), workstation graphics (FireMV) and workstation RAM (registered, ECC). Indeed, the mac mini has a laptop CPU and SO-DIMMs for memory, but we're talking about the Mac Pro.
Furthermore, I don't get the "doubts about the thunderbolt displays". Thunderbolt can act as a simple mini-display port (with audio also). So go grab your $150 Dell Display Port monitor and plug it in. All it takes is a $8 mini-display port M to display port M cable. If you want to use the more advanced features of thunderbolt, it's a matter of taste, but for a lot of external hardware USB is not an option even in it's 3rd incarnation.

Stop scratching the machine if you don't want scratches on it. My workstation is always on, and I think that except for dusting it, I haven't actually touched it in over 1 year. Now going to serious stuff...
Upgrades are allowed. It features 6 Thunderbolt ports so you can add as many 10GigE, FC, HBA, high performance external directly attached storage arrays, Video Capture controllers as you want. There are a few thunderbolt to pci-express 2.0 8x adaptors available if you want to use your own hardware.
I guess that the only non-upgradable parts are the video cards. I think that they are swap-able but due to their proprietary format there would be no 3rd party alternatives.

P.S.: Have you noticed how Google managed to come up with a decent Maps app in only 6 months? They completely neglected the iOS distributed app for years and only improved on Android until Apple kicked their arse back to work. I find that kind of competition to be healthy!

If you were Apple, you wouldn't have survived the 90's.
While the Apple maps data is not the best in some places, I can say that they're doing a much better job improving than everyone else. It took Google a few years to have any roads listed in most European countries. Apple started with complete maps. I've compared the coverage of Apple, Google, Nokia, Bing and OSM on quite a few occasions and OSM is the only one better than the rest. Google, Apple, Nokia and Bing are not showing one third of the motorways in Romania. I'm not talking about a forgotten secondary road somewhere up in the mountains, I'm talking about (albeit a few) hundreds of kilometers of motorways.
The application isn't bad at all. It's still superior to Google's, at least for now. The data might be flawed in some places, but you should give them a few months to get it right. I'm quite sure that when Google Maps first appeared, their data wasn't optimal either. Their maps are now much better due to community effort in apps like mapmaker.
In case you're an idiot and couldn't figure this out by yourself, I'm going to spell it out: it makes perfect business sense to build your own maps application if your biggest competitors (Google, Microsoft, Nokia) all have their own solutions. What do you think the licensing costs would be if Apple attempted to license a maps solution from Nokia's Navteq or from Microsoft's Bing?

You haven't been to the Netherlands recently. NS should stand for "No Show"!
In my experience, while traveling between FR, DE, BE, LX, CH, AT and the NL, once a train (including a high speed train) crosses the Dutch border it's instantly delayed. Should I count the part where they are changing the trains to between NL and BE to "high-speed" trains, even if they are traveling at normal speed, is just an excuse for making the prices 3-4 times higher and with mandatory reservations (unless you buy the tickets from Belgium). Should I count the times that I've wasted on their platforms mostly in bad weather.
The Dutch are good at a lot of things. Punctuality hasn't been one of them in a long time, whether you're talking about KLM, KPN (especially Getronics), NS they have completely forgotten what punctual means. Furthermore, they have replaced their BS-free attitude to a disgusting "politically correct/tongue up your arse" attitude, where, in order not to loose your business they tell you what you want to hear instead of the ugly truth. Fortunately, the Germans and the French are still frank enough.

You didn't read what I said. Yes, ZFS+Snapshots, but you also need at least Sun Cluster replication and tape backup. ZFS + Snapshots doesn't save you from fires, floods, software bugs and ill-will. It does save you from idiots, and disk failure though.

RAID is a method of reducing the chances of a disk failure being fatal to the data. RAID is not a backup solution. Anyone who answers a question about backup with RAID is an IDIOT who doesn't deserve his oxygen quota and should be put down.
Disk failure is not the only reason for using backups. More often than not you run into an idiot user (who happens to be executive) that deleted stuff by mistake and you need it back.
Furthermore, disk failure can happen on all the disks at once. You have: fires, idiots, floods, more idiots, bad wiring, idiot admins, software bugs, and my personal favourite: tired admins.
Always have an off-line back and an off-site replica is my personal favourite.