Because the campaigners always take it a step further and say it's more distracting to be on the phone than not even if you have a hands free kit. Usually their studies are in contrived circumstances and omit that doing anything other than focusing 100% on driving is going to increase the risk of mistakes, including just having a conversation with someone sat next to you. Are they going to campaign against car sharing lanes as they encourage people to be in the same car as each other which in turn promotes distracting conversations?
Of course not as that would be ridiculous. Likewise banning hands free mobile phones may improve accident rates slightly but doing so is absurd, and yet that is what people are campaigning for. Here in the UK we already have enough laws to cover this - using a non-hands free mobile phone whilst driving is illegal, and driving without due care and attention covers any other poor driving due to distraction of any kind, but requires you to actually be driving poorly. We're not all the same and there are plenty of god awful drivers out there who are quite oblivious to what's happening around them regardless of how much they are forced to concentrate on driving the car.