Well, your options boil down to three (or four) choices.
1. You own your data and control its access entirely. Every time physicians, clinics, pharmacists, researchers, etc need or want access to your data, you must authorize them (to whatever extent you wish, for however long, etc). This feels like the holy grail of data access and privacy, but it also puts the legal culpability entirely on you. Give someone bad access? You're responsible. Lose the data/access device? You're responsible. Forget to bring it to your visit? You're responsible. It's like carrying around your medical data like cash, it's irreplaceable without a lot of hard work, vulnerable to theft or misplacement, but affords you the most tangible method for control.
2. Your data is held in escrow by a third party. This would be like a hybrid of the above and the system we have now. Imagine that the store you shopped at also held your bank account. Obviously, that sounds like a recipe for disaster. Our banks and credit systems are the escrow parties for our financial means (or you could use cash as in option #1). A similar system could be adopted for medical data in which hospitals, clinics, pharmacies, etc must plug into a third party in order to access your data, by your control and authorization. It creates one more link in the chain, which can aid (or also detract) in security measures, decrease personal liability (if someone steals the data from the escrow party, you're not liable and can sue for damages), but also probably costs a fee for access to your own data, either by you or the clinic.
3. The government acts as an escrow party. Enter the libertarians and anarchists to rip this option to shreds.
4. The clinics own your data and share it with others/copy it to you upon your request or authorization. The status quo.