As someone who does this for a living, I always feel like fairness, balance, and skill are just tools. The thing that really matters is player psychology. The game doesn't actually have to really be fair or balanced. It just has to *feel* fair and balanced to the player.

Skill and balance are a means to that end, but I don't think they should be the primary focus.

It's tempting to think just making it perfectly balanced will make it fun, but that's only part of it. If the player experience isn't considered, it's easy to make something perfect mathematically, but boring or frustrating to players.

As Jeff Kaplan (former WoW lead) put it, Perfect skill matching would mean you lose half your games. Is that fun? Is there any other way to do it and be fair? Probably not. So they found another way to make it feel fun anyways.

Whenever I've played with friends who were used to RPG-type games where they're always the hero, they get confused when they play against real people and die a lot instead of easily killing everything like they did before. That makes me think designers should think about how they transition these players from killing computer opponents to real ones.

As far as measuring skill goes, a perfect skill-based system would determine your skill quickly and accurately and it wouldn't change much. That would feel pretty boring and probably wrong to a player.

So if your goal is to make a game fair and balanced, you're probably barking up the wrong tree. Instead, think of fairness and balance as some of the tools to make it fun, and focus most on the perception of the average player.

The counter-example would be Consumer Reports. The problem is people would have to be willing to pay for a subscription to read "unbiased" video game reviews. It might be hard to get people to pay for them when there are so many free ones and "meta" review sites like Game Rankings and Metacritic.