I would be ok with removing all tax exempt statuses from churches as long as charity work was deductible for them. We could then see which churches really do put their money where their mouth is in charitable work and donations. It would also encourage any religions which don't put much effort in helping the poor to change that behavior rather quickly.
If we pay our taxes why should we pay again, sometimes twice (through charitable donations to and tax breaks for "charities") for services we paid our governments to do in the first place?
If there is any charitable work that governments should not pay for then they shouldn't pay for them, whether by tax breaks or allowing deductions for what are portrayed as "non-business" expenses.
Governments should use money from taxpayers only for what it is meant to be used for. Tax breaks that governments give charities allow either charities to profit from the government abdicating its responsibilities or force me as a taxpayer to subsidise charitable causes that I wouldn't by choice and cannot be expected to under any "social contract". After all, if I can be expected to pay for any cause under a social contract then society as a whole should - and that's tax.
Lead me not into temptation... I can find it myself.