We've also been trying to wipe out lions, tigers, bears, and most other predators (oh, my!) for thousands of years, so that must make it ok to drive those animals into extinction by killing every last one of them now, right? Just because we've been doing something misguided for a long time, doesn't make it ok, especially now that we're much more efficient at it. That being said, many GMO modifications aren't substantially different than those achieved by hybridization and selective breed, which has itself resulted in pretty massive changes in plant and animal species over the years. Ok, so at what point does modifying a species become a bad thing? That's the problem; it's almost impossible to know in advance. We've survived for millions of years as a species precisely because we don't agree on everything, it is useful for the survival of the species to have a certain percentage of people that disagree and refuse to do the "obvious" thing, because in a small percentage of cases, the obvious thing is actually fatal. For example, Christian Scientists refuse to get blood transfusions? They don't get HIV or many other blood-borne diseases. The lunatic fringe is actually ensuring our survival as a species.