This only works as long as everybody is equal.
Precisely. And since, in terms of economic strength, everybody in the expanded EU most certainly isn't equal (please note that this is not intended as any sort of insult, merely a statement of fact) the free movement principle does not work well.
In particular, what has really happened in certain cases, for example with Poland and England, is that most of the movement has been one way. This puts strain on English services, but it's important to recognise that it also means many of the people who would be best placed to help Poland develop its own economy are among the most likely to find working in richer European countries more attractive and/or lucrative, creating a "brain drain" effect back home. In the long term, both nations could end up worse off because of the imbalance.
In principle, freedom of movement is a good idea, for both business and pleasure purposes. But on the business side, it does require reasonably balanced parties so the traffic at least roughly cancels out. This was the case in the early days when there were far fewer nations in the shared European machine, but with the expansion to nearly 30 actual or aspiring member states with much more diverse economic conditions, the same logic no longer holds.