I'm not a sports fan myself, though I might be something of a meta sports fan in that I find myself intrigued by the phenomenon so to speak. I liked the movie Moneyball for instance.
So, I reckon some people probably like the fact that boxers are actually hurting each other (and some of them may be appalled by pure torture, and only like the act when it is between consenting adults), and others merely like the conflict, while still others genuinely appreciate the athleticism and skill of the 'sweet science'.
There's something primal about a fight, a no holds barred struggle to decide who is best, who would triumph in a desparate primitive situation. The problem is the damage of course. In a no holds barred with eye gouging and choke holds even the winner may come out with a permanent injury, so they have to have rules, and then it's a matter of who is best at working within the rules.
Some people like the fantasy version, professional wrestling with all its kayfabe. I'm sort of intrigued by the idea of robot fighting where it's OK if one contestant is totally destroyed, but from what I've seen, the technology is still too primitive for me to stay interested.
Sumo wrestling has a certain appeal because the individual bouts are very short, which makes it a little more viewer friendly than standard western style wrestling (legit wrestling that is), though the ceremony of entering the ring, scattering rice, glowering at each other while waiting for the ref to start the match, can take awhile. Also, as far as I know, serious injuries in Sumo are rare, but Sumo wrestlers tend to have short lives because of their diet. I sometimes think someone should design a form of combat where the matches tended to be very short, like Sumo, but designed to favor people who are all round athletic, but also with little chance of serious injury.