Actually, outsourcing and importing for cheaper is part of what gives the poor a better quality-of-life. The unemployment argument is a red herring: bringing any trade jobs back incurs an added cost, which raises prices and thus eliminates other jobs. If those prices raise enough to diminish the newly-created jobs beyond the number of jobs lost, then you have a net-loss of employment; this means American workers producing trade-import goods would need to be paid little to net-create jobs.
Even then, the change is disturbing.
Right now, Men and Boys's Cotton Trousers and Shorts retail for an average of $14.97 per pair (this is a rough Google number, and is probably inaccurate; it's also the only factor that can be variable and still correctly-demonstrate the principle). The Chinese import cost is 6 cents per pair (40,000 pairs shipped in a 40-foot shipping container, at an import cost less than $1,300 from China to US), with the Chinese labor cost at $6.14 per pair (via the published total number of imports of MBCT from China PRC and the total cost of those imports at import time). The difference in import and price includes the domestic shipping (truck drivers), retailing (inventory associates, cashiers, managers), logistics, and infrastructure (power, maintenance, rent) involved in local sale, as well as the profits.
If we paid American factory workers above $18/hr to make MBCT, with a retail average of $14.97, we would lose total American jobs; if we paid under $18/hr, we would gain jobs. This is because the cost of MBCT would increase, and the total purchaseable goods would thus decrease, impacting the entire logistics chain of shipping and selling them, as well as reducing the number of factory jobs to make them; and the factory jobs recovered from China are added to the job market, offsetting this. If more jobs are lost than gained, you lose jobs in total.
That's not the issue.
Say you pay your factory workers $21/hr, the same salary as a GM line worker. The price of MBCT goes up from $14.97 to $50.57 (remember: $8.83 of that goes to American wages for cashiers, truck drivers, shelf stockers, and the like, with some carved out for taxes and profits; I'm assuming profit margins and taxes fall instead of increasing as well, instead of adjusting that $8.83 larger). Today, a $21/hr income lets you buy MBCT at 0.71 hours's work per pair. With $21/hr factory workers, they'd work for 2.4 hours to afford a pair.
If you pay them an $8.25 minimum wage, the price rises to only $25. Today, an $8.25/hr wage lets you work for 1.8 hours and buy pants; at $25, an $8.25/hr wage requires you to work for 3.0 hours to afford the same pants you're making.
If you think that sounds ludicrous, consider: before globalization, the median American household spent 12% of its income on clothing; once we started outsourcing to China, this rapidly fell to 4%. It's now under 3.5%--it's only slowly continued to fall since the great globalization revolution. That means globalization in fact decreased costs to 1/3 what they were.
Outsourcing your jobs to another country that does the work a hell of a lot more cheaply creates an enormous capacity to buy, but somebody has to transport all that shit you're buying once it comes off the docks. You can't sail a ginormous shipping friggate up the mid-western basin to Colorado. That creates local jobs. Even then, unemployment dips nice and friendly-like, but it gets buffed out as population expands to fill the abundance of jobs, until some factor of scarcity (job scarcity, food scarcity, etc.) creates an expanding population in poverty and slows growth. Likewise, a small loss of jobs slows population growth until it adjusts to fit its economy's capacity, and can have a profound effect on the size of the labor force.