To be fair, a PC from 2009, like an Athlon II, can be indistinguishable from a 2014 Core i3 when you're running, say, Windows, Office and Firefox. By comparison, an ARM processor from 2009, like the one in an iPhone 3GS or Motorola Milestone, is frustratingly slow even to browse simple websites. Pit against a Nexus 5 to have an idea of how much faster the pace of phone processing power is happening at than tradicional x86 right now.
Also, I'm pretty satisfied with my Nexus 4, except for heat and battery life. If I could exchange its processor for a quad-core cortex A7 like the one inside a Moto G (the damn thing lasts over 5 days with light use), I'd be happy to do so.