I don't like the idea in theory, but in practice I have to agree with you - if people disagree, they should post. Yes, that might mean they can't moderate in that thread, but so what - if they want to say something, they should say something. Participating in the discussion in much more important than moderating it.
But people don't do that - they moderate something as flamebait simply because they disagree with it.
Perhaps a better answer is for people to be able to moderate moderations - click on the score, see how it was moderated to the value it has, and be able to check negative moderations as "unfair."
In any event, "disagree" mods should not be allowed to drop a post below 1 for a user, 0 for an AC.