Your title does say it all.
America is saturated with unskilled labor. Those people need to live somehow. Until we have some kind of guaranteed base income or some such system, they need a job.
Aside refugees from extreme conditions, it's difficult to me, morally, to "help" massive amounts of semi-skilled foreigners when our neighbors might be wondering where their next meal will come. They should come first. The whole "you can't help others if you can't help yourself".
We can't save the world. We just can't.
Now that doesn't mean these work visas should not exist. There is totally a need and large benefits for having visa programs for high skill immigrants where there's real needs. And stuff like H1B should be exclusively used for that.