Or even better, zero family or friends.
You know, I think this might be key, especially the family thing.
The 2 people I know who are in their 40s with paid-for houses, good investments (above and beyond 401k, etc) and lots of savings are REALLY cheap people. Relentless coupon clippers. Buy a huge cut of meat at Costco, cook a giant stew and eat it for every meal for a week. Vacation is staying home from work 5 days to paint the house. Can do everything short of an engine rebuild on their car (which they have owned outright for 7+ years). Only watch movies they buy used from the pawn shop. Clothes all bought at discount stores.
And neither one has much of a social life and no spouse or girlfriend.
I don't think living that way would be that hard, but getting other people to put up with it would be. I think women kind of generally look at spending behavior as a kind of signaling -- how well will you take care of me -- and if they see a guy who won't spend on himself, they figure no way, he won't take care of me or will be unpleasantly cheap.
The only *families* I've ever run into that cheap are super religious, scrimping so mom doesn't work or some other kind of lifestyle goal. And I don't think they really are accumulating anything, they just don't have anything because of one income.