The 47% figure is rather misleading because it only refers to the percentage of people paying federal income tax. It turns out there are multiple federal taxes on income, only one of which is called the federal income tax. Most of those 47% pay the payroll tax which is a regressive income tax. For detailed numbers see this chart which Google image search found on this CNN Money video. For those that don't want to click the link, the breakdown according to CNN is 53.6% pay income tax and the rest not paying income tax are split up as 28.6% pay payroll tax, 10.3% elderly with no income tax, 6.9% non-elderly with income under $20,000, and 1% other.
Once you eliminate people paying income/payroll tax and the retired elderly, that leaves at most 8% not retired but not making enough money to owe federal taxes. Some of those are unable to work. Some of those are unable to find a job. Some small proportion might really be lazy and leeching off the system like you are worried about... but that is almost certainly much less than 8% of the population and definitely a lot less than 47% of the population.
On top of that, remember this entire discussion is only about federal taxes. There are also state taxes, which are pretty universally regressive. Particularly, most states have a sales tax which hits the poor much harder as anyone earning so little they aren't paying income tax is probably buying necessities with all the money they do earn and therefore immediately paying sales tax on a large proportion of their income.
This article that I came across while searching for those figures tells a similar story with more exposition and citations.