A response I've seen to this argument is, "the majority of the population don't murder and rape. So, we don't need those laws on the books about these crimes then?"
Think of it this way. People fall into three categories for any given action:
Group 1: Will not do something whether its illegal or not.
Group 2: Will not do something if its illegal. If no law exists, they will.
Group 3: Will do something whether its illegal or not.
Laws aren't needed for group 1 and don't affect group 3. So really group 2 is what a law is focusing on. The people that fall into group 2 for murder and rape is small but the number is non-zero. Keeping and enforcing laws against those things helps keep people safe by deterring the people in that group.
With gun laws, again, group 2 is again the only group affected, but group 2 is only a danger if they fall into group 3 for a more serious crime, which is virtually never going to happen. Nobody who is going to commit rape/murder/robbery/etc is going to have second thoughts about breaking a gun law. Ergo, if you're in group 3 for a violent crime you'll be in group 3 for the gun law and hence unaffected by it.
Its often said that "Well, gun laws at least make guns harder to obtain.", but as I said earlier in the Unites States there are more guns than people. We rank #1 in the world for guns per capita. From Gunpolicy.org the ratio of guns to people here in private hands is 101.05:100, with 34.4% of all households containing at least one gun. Estimates are 110 million rifles, 86 million shotguns, and 114 million handguns. There is no mandatory registration in the US. You could declare gun ownership completely illegal tomorrow and there's still enough guns that it'd be dirt simple getting ahold of one for over a century.
Even that is ignoring the fact that with things like 3D printing guns are getting easier to make at home, and even before that they weren't rocket science - people have been making some form of hand-held gun for ~800 years or so with pretty basic tools, and even what we'd consider to be a fairly modern gun can be made with simple and cheap tools (drill presses, files, etc).
You're simply not going to put the genie back in the bottle here.