Equal treatment under the law is very much a rights issue.
Consider: only white people get food stamps. Sure, that is "just a benefit, nothing more, nothing less", but it is obviously racist.
At one point in our history, many states forbid black white marriages. And sure, that marriage certificate was "just a benefit", but that situation is obviously one about Civil Rights.
The Polygamy argument is just silly. It is the slippery slope argument that most far right conservatives start with, and it usually ends with them mentioning something about bestiality. However, the slope really doesn't exist. Polygamy has historically been shown to be a bad idea. And one that is nearly always about the exploitation of women. The split second that women get close to equal power in a society, polygamy vanishes. If polygamy is a non-starter, the slope doesn't exist.
Marriage has always been about two people creating a partnership, which creates stability and benefits society. As our society has changed, grown, and matured, we've decided to code into law protections against discrimination. At first, they were obvious ones, like race, religion, etc.. The latest one happens to be sexual orientation. It just takes time for a new civil rights protection to seep into all the aspects of society that people use to inform their worldviews, like churches for instance.