The right granted by society when they formed a government.
The short of it is that most people want some kind of government. Most want at least some police capability, somebody to come put fires out, build roads, and that sort of thing. Those things cost money.
You were probably born into a nation that existed for a long time, so you didn't necessarily grant consent personally. You can leave. I understand that can have some financial costs, which I personally disagree with. If it were up to me, I'd let anyone renounce their citizenship at the border on their way out for free.