The interesting thing for me is we have a compute cluster running OpenGrid Engine (SGE). It seems like one of the best fits for cloud computing at first blush. Much of the time it's idle, so "in the cloud" it costs "0" when not using it. (there's always an OS image storage cost or something so keeping it ready to go isn't truly $0 but closeish)
The problem is data access. We have tens of GB of data and a 30Mbit pipe at best to the cloud. The compute is worthless if they can't access the data, and local 1Gbit has too much latency for network operations - it can cause orders of magnitude slowdowns in waiting for the NFS transactions to occur. For certain data sizes, you can locally RSYNC the data in for processing and then RSYNC out your results. But if it takes 4 days to upload the dataset - even if you could use 10x as many nodes, you're still done sooner on site because the data transfer is 1Gbit instead of 30Mbit at best. And on site you can limit to where the transfer is 10Gbit (some nodes support it).
Latency, difficulty of accessing local environment, accounts, network storage, throughput limits all ON TOP OF constant billing which is usually not any cheaper TCO compared to lower maintenance costs on owned hardware and depreciation.
Heck, even for a simple tiny multi core process, we tried cloud and it went no where. We could get a 16 core machine for $2 / hour. The problem was we either had to pay for the whole month, or there was a variable 10min-1hr startup time before you could then log in to the machine because shutting down the VM didn't stop billing - you had to decommission it, and recommission it each time, which meant waiting for the whole OS to load over the SAN at the cloud vendor.
Our users found it cheaper to buy a 8 core machine than pay for 3 months of the cloud service. And they refused to wait that long each time they wanted to use the software - they'd get distracted and never run the calculation with that lag to starting.
Even e-mail is crappy in O365 for us. We went from broadly supported IMAP standard e-mail, to whatever MS thinks is IMAP, OWA and Outlook. All of which end up worse for most of our users, but especially vexing for the Mac and Linux users.