Sounds like bad loans to me. It most definitely was people buying houses they couldn't afford, although you can make the argument the the average person is too dumb to know if they can afford a house or not so the size of the loan they can take out should be limited.
The banks that issued those loans played some games, but other banks and insurance companies bought it. They should have known better. If I tell you I've got a AAA bridge to sell, do you buy it without doing a little checking? Either way, everyone involved should have lost their money so that they'd be a little less trusting in the future. If you're investing money you should know what it's invested in, what the risks are, and who's managing it if it's not you.