I think there are plenty of perfectly nice large and strong people to handle the tiny few who suddenly decide to go rogue.
Violent people very rarely "suddenly decide to go rogue". They typically have a history of increasing violence.
The odds of one of those perfectly nice large and strong people being around when one of those rogues attacks you is low; and firearms enable people who are perfectly nice but not large and strong to protect you.
A quick internet search turns up many instances of ">older ladies defending themselves with firearms, and of mothers defending themselves and their kids, when no nice young large and strong folks were around. People who want all guns to disappear want a world where those ladies would be defenseless against their attackers.
Hey, I already walk around unarmed --- a short, flabby weakling --- and yet don't regularly get beset by burly bandits.
Nor are you regularly shot at by gun-toting thugs.
Part of the reason that, if you live in a middle-class area, you're not very likely to be assaulted is because we hire people with guns to lock up burly bandits. And also because potential burly bandits know that some portion of potential victims are armed -- armed citizens create a penumbra of protection.
With guns, I'm still at the mercy of those better armed, with better marksmanship, and more willingness to initiate violence with the element of surprise
All of those factors apply even more without guns (except substitute "marksmanship" with more general "skill"). In a typical assault scenario, if you have a gun your attacker is not significantly "better armed" even if he has a bigger gun, and marksmanship is not much of a factor because assaults happen at close range. With knives or other weapons, both the size of the attacker and of the weapon matters much more, and skill is a huge factor.