Their major blunder with the PSP/UMD was to redesign an existing concepot (the Minidisc) as the UMD to differentiate between the new shiny better one and the old.
Thing is, the Minidisc had a minor but loyal following, and in 2004 got a major upgrade with the Hi-MD format that allowed data and video to be trasnferred on top of music. And the major advantage of the format compared to the newfangled UMD was that it was rewritable.
If they had released the PSP with Minidisc games, videos and whatnot, I'm sure the console's story would have been completely different. Even with the Memory stick slot on the side. Both rewritable formats, and they'd have been SONY so presumably they wouldn't have lost anything. Of course that would have meant trusting the customers with an relatively open media, and that's something they're allergic to.
Instead they created the UMDs, closed and crippled them, and tried to sell them at the same price as full blown DVDs. No wonder it didn't take off. Meh.