The short version of the problem is that the level of service you can expect from each system is incredibly variable, so it's hard to offer a meaningful QoS for the system as a whole. It's not quite as bad as the distributed-hash-table problem (a.k.a. P2P file storage), but it's still bad. (Zooko once told me that MojoNation saw an average 50% turnover in nodes in a 24 hour period.) But it's also not as easy as having all your distributed nodes dedicated to just storage, and even that's a really hard problem to solve. (I should know; my company is one of the few vendors doing it.)
Someone else suggested OpenAFS. OpenAFS is fantastic, but not for unreliable server environments. I really don't think there's a complete solution out there, but not for lack of asking.