You can turn off overcommit in Linux if want to - most people find the default behavior more useful since many applications allocate memory they do not need to use, and don't handle out of memory errors gracefully. Change the sysctl "vm.overcommit_memory" to 2, and see "Documentation/vm/overcommit-accounting" in the linux kernel source for related sysctls.
There are occasions where you might want to use a lot of swap, if there are one-time-run applications that use a ton of RAM to do something like image manipulation/scientific computing/whatever. Those might be rare, but it would be very irritating to get out-of-memory errors just because the kernel doesn't feel like using swap.
When the OOM killer is invoked, the application isn't usually allocating memory - it's using memory that it has allocated before and that the kernel overcommitted on. So there's no good way to send an out-of-memory error other than by something like a signal handler. I think the reason this isn't done is because the signal handler would likely need to allocate RAM to run (maybe to get its code paged off disk) and this wouldn't help with the memory pressure.