Don't think of it as a replacement for a console, but instead of a way for you to take your console game out with you wherever you go. That has the potential to be a powerful combination. A full console game you play as normal at home, but while you're out you can still work on character progression, or play with your friends.
Sure, the gameplay experience on the phone isn't going to be identical, but it doesn't have to be. The added convenience of having your games with you wherever you go means that people are more than happy to put up with a smaller screen. I was a huge fan of flight sims on the PC, now one of my favourite games is a space combat sim on the iPhone. Sure, the graphics aren't as good, and the controls are a bit more fiddly, but it has the all essential essence of the game, it's fun to play, and because I always have my phone with me, I can play it anytime I want.
Sure, this might not be for everyone. For a die hard console enthusiast, with plenty of time to stay home and play, they're always going to prefer native console games. However, there's a much bigger market of people for whom gaming is something they have to fit around their other commitments, and for that market, this could be huge.
Also, if your phone becomes an integral part of the games, imagine this: There's nothing to stop companies using phones as a wireless controller with a built in display, and built in storage. You can use the phone to display game elements distinct to your character, and to store your save games. So now you don't need to buy consoles with a bunch of controllers for multiplayer gaming, if you want to play with some mates you can head over to anybody's house with a console, you all have your controllers with you, and you all have your saved characters. Plus you could start the game while you're in the pub, not many consoles can do that :-)
And I'll leave you with one final bit of food for thought: Imagine what's going to happen if something like WoW were to adopt that platform... An immersive MMO that you can play online at home with your friends, or play with friends at somebody's house, or just play on your own anywhere you like...
I doubt phones will replace consoles, but there's potential for them to supplement them beautifully.