AFAK The original design of the ps3 didn't even have a GPU, the cell is more than capable. Those 10,000$ IBM cell blades, they are designed for high level graphics processing, to be sold to the likes of Pixar and such. (Note: I do not know if Pixar is using them, its an example.) I'm not sure why Sony ended up going with Nvidia GPU's. Possibly because they were already late to the game, the game dev's were pretty pissed off about having almost nothing they could port easily, and Nvidia was the compromise.
I had Linux in mine, with a bit of hacking you could get the frame buffer to run out of the GPU's memory, witch made the memory crunch suck less. Had Other OS remained I'm sure some one could have built a 3D driver on top of a couple of the cells. I read rumours that a plan was being hatched before Sony pissed on the fire and raised a stink.