Rights do not disappear because you associate with someone, or because you have more money than them. Rush Limbaugh has just as much right to free speech as I do, despite the fact that he influences a great many more people.
Actually rights can and do disappear because you associate with people. The entire crime of conspiracy is premised on the idea that multiple individuals working together as a group are more dangerous than the individuals working separately. If I say "I am going to rob a bank," I've done nothing illegal. If I say to Person A "Do you want to rob a bank with me?" and Person A responds "Yes," then I've just committed the crime of conspiracy.*
In criminal law it's viewed as suspect when multiple people combine forces, and I don't see why this shouldn't continue to be true in regulation if those people incorporate themselves.
*Yes, this is a very simplified description of conspiracy. The wiki article has much more detail: http://en.wikipedia.org/wiki/Conspiracy_(crime)