Yes and not long ago all US citizens had the right to be married to one person of the opposite gender of the same race.
What the rest of the society considers worthy of recognition as "marriage" should, indeed, be up to that society... This is not about sex-life, which should be up to the willing participants, but about other people's opinions: "We are a married couple!" "No, you are not — you are two men..." "Yes, we are — this new laws says so!"
To force others to consider a particular union as "marriage" may be illiberal...
No doubt eventually we will knock the gender caveat off of marriage too
Sure. And move straight on to fighting the species caveat next, will you not? Or, perhaps, the one person caveat will be next — why can't I be married to two consenting adult human females and an adult male cat — at once, after all? How is that prohibition not discrimination?
While we are at it, lets just get the state out of marriage and just allow exclusive legal partnerships between any two consenting adults.
Full agreement here — there should be no special provisions for "spouses" in any laws — we are all equal subjects and citizens. Until then, however, "marriage" and "spouse" are legal terms and their precise definitions are up to the legislatures...