It's more than just that, though. A mouse allows you to represent a huge space of possible pointing positions as a 2D plane, which we're very good at dealing with. For FPS games, you can make any aiming movement from twirling around 180 to shifting your aim slightly in an extremely short amount of time by moving the mouse to a distance on the plane that represents that angular shift.
With a controller stick, your control over aiming is a matter of holding the stick at a certain angle for a certain amount of time. The angle of the sick is much less easily controlled and the time you have to hold it has a direct impact on the player's ability to make twitch movements. Adding aids for controller users like smoothing the input (to make the stick angle more forgiving) or adding acceleration (to cut down on the time it takes to make a large movement) only makes the controller less fit for rapid precise aiming.
Of course, a keyboard having more buttons is nice, even if they aren't in really the best layout for gaming. Being able to directly switch equipped items or activate certain items helps. Avoiding the overuse of "action buttons" makes it easier to control what your character is actually doing instead of accidentally doing something you didn't want to do (although these are usually carried over in console ports).