USB isn't built for juggling machines. Every time you switch from Machine #1 to Machine #2, the USB devices all magically vanish and disconnect from Machine #1 (prompting drivers to be disabled), and magically reappear on Machine #2 (drivers now have to be loaded and initialized).
Good KVMs show up as a composite HID device and always present that to the OS, even if a different computer is now active. One of the things this means is that most gaming mice and some keyboard keys aren't passed through.
Really good KVMs have an option to disable this feature and allow straight pass through of the devices to allow all functionality.