The real numbers:
If you figure out what the "just enough" solar panel count would be for your max power needs during the shortest day of the year in full sunlight at whatever angle(s) you'll be able to manage, you'll need five to seven times that many to make sure that on non-sunny days you're good to go.
This is because solar panels produce between 15% and 20% of rated capacity on non-sunny days. Non-sunny days are not "dark", they are only "dim." It is a very rare day indeed where it is so dark as to drive a solar panel below 15% output (major snowstorm which has the atmosphere nearly opaque, that kind of thing.) But dim days can come in very long strings, so that's the target to aim at.
For a reliable system that will never let you down, you do tend to need considerably more space than you would initially think. But it is possible, given that you have the space (lots) and the budget (also lots) required. Panel-wise, it's a quantity issue.
But there's a wolverine in the woodpile: Batteries. To be blunt, they suck, as in, expensive to replace and very short lifetimes compared to the rest of the system.
Until or unless ultracaps reach a point where they are on par with batteries for the service you need, reasonable full-on solar installations remain quite expensive.
Installations that use batteries are regarded highly by their owners only until the first battery replacement. Then their wallets straighten them right out.
I have a lowish-power setup, with an unfortunate number of ultracaps (because capacity is very low, about 1/10th that of a battery right now) as the energy storage medium. I did it both to give my ham gear a constant supply, and to explore what would actually work. It took a custom controller design -- ultracaps don't act even remotely like batteries -- and it took me quite some time to put it all together and make it work like I wanted it to. There are way more panels than you'd expect because of that 15% number (my panels are cheap ones), and there are way more ultracaps than I wanted to expend room for, but I did have the room, so I kept at it. It works great, and it isn't going to need service for decades unless there is an actual component failure or a severe weather event (large hail, for instance.)
Trying to do this for a full house load? A typical US house? Not yet, I'm afraid. There will be tons of compromises to make in appliances, lights, and lifestyle, and in the end, you're not likely to have the same lifestyle you had prior to your switchover.
The battery problem will probably be solved. One way or another. Eventually. I have no idea if solar panel efficiency will get up into a range where the costs and space will fall within the range needed to go truly off grid. That's a physics question in an area where I have nowhere near enough knowledge. But right now? No.
Which brings us to on-grid, grid power use mitigation. Now that is an interesting area, and we can leave batteries right out of it, as peak power also comes during peak power use (right now... electric cars may change that.) But it involves all kinds of compromises for the utilities if it is adopted in any kind of mass manner. They will need power storage, as I understand it at the moment.
Full-on solar is a great, great thing with huge potential (ha, a pun, hooray), but it's not a panacea by any means except in very rare sets of circumstances that involve very large amounts of money and large areas of space for the panels.