I own a Samsung Galaxy Note and I knew beforehand how large it is. But I didn't buy a mobile phone, I bought a small tablet with mobile Internet connection and that's exactly what I got. Yes, I can also make phone calls with it, but I rarely do.
As for carrying it: There are nice belt pouches for it.
The solution is simple: If you don't like it, don't buy it.
Of course, if somebody can come up with a smartphone that has a >=5" display but is much smaller, that would be nice. Maybe with glasses? That would still leave the problem on how to interact with it (no, speech recognition is not a solution). We'll see.