Dragon actually is man-rated and has actually had people inside it, while in orbit and attached to the ISS, without killing anyone. It's just not a man-rated *launcher*, which would require a launch escape system, various additions to support people, etc. The requirements for man-rating Dragon 2 and the Falcon 9 are more extensive but not overwhelmingly different. They've already had people bouncing around inside the Dragon while in orbit, there's no reason to think they won't get this done.
And 12 launches without a single loss of vehicle or failed primary mission, and one partial failure of a secondary mission due to ISS safety rules is hardly "an abysmal safety record". It's arguably a better start than either the Atlas V or Delta IV had...the first 12 launches of both of which included a partial failure that left the *primary* payload in the wrong orbit.