TIP: Penn&Teller: Bullshit! episode 2-5: recycling. That should get rid of some of the false preconceptions that you have.
Also, in our past we (the western countries) have dumped lots of toxic materials in our own countries and we have managed to either contain most of the problems or even clean up the affected ground. Current dumps are pretty safe and bound by strong regulations. The 'poor' countries are simply a few decades behind, but they're catching up quickly and will take care of their own backyard soon.