Maybe because "we" north European countries are part of the European union which makes it easy for people to travel between poor and rich countries.
Note that I don't think it's the poor peoples fault. There's obviously some sort of organised crime utilising poor people for their own gains and this is made easier just because of the ease of travel, but that same ease of travel is a big win for other parts of society, and it's also fair to not bind people to the geographic region they happend to be born in. I'm sure americans have similar problems.