Reprising a previous comment:
While the universe of banks willing to accept high-risk merchants is smaller than the total number of Visa association affiliates it is certainly far larger than three. If you got these three banks out of the game, there would be others to replace them. However, the more important asymmetry here is not in the size of the set, but in the switching time. If a merchant (or their payment processor more likely) starts to route transactions through a new acquiring bank, their identity will be revealed very quickly in any purchase authorization record. By contrast,the time to actually establish that new banking relationship (and get appropriate certificates from Visa, etc) takes days. This is one of those rare cases where the defender is able to respond far more quickly than the attacker.