Oh? So we should allow all citizens to tinker with high-yield nuclear weapons in downtown metropolitan areas, since there is only the *potential* of harm provided they don't do anything stupid? I think that's going to be a hard sell.
Or how about juggling sealed vials of weaponized Smallpox at the World Fair or something? So long as you don't drop them there's only the *potential* to kill hundreds of millions of people, so it should be perfectly legal, right?
I agree with you in general principle, but there's a *really* strong case to be made that if the potential damage is great enough, then even the potential warrants restriction. And once the door is open then we have to decide exactly where the line is. Firing a single handgun into the air on New Years eve is unlikely to do any real damage, but if thousands or millions of people are doing it all at once, then the odds that at least one of those bullets will come down on someone unprotected with enough speed to injure or kill them approaches 100%.
Sadly, history does show that such thinking tends toward a slippery slope but I am forced to argue that it is still justified in some cases, and that we must stand eternally vigilant in the grey areas rather than succumbing to the tempting simplicity of black and white thinking.