That conventionally-bred gene manipulation you mention, while resulting in similarly granular effects to that of the GE, has the benefit of using mechanisms and pathways which have stood the test of those 2 billion years without resulting in catastrophic species loss or damage - *that's* why it gets a free pass .... in my book, anyway.
What about chemical or radiation-induced mutagenesis? I work in the ag biotech industry and I can tell you with 100% certainty that at least some of the cultivars marketed as clearfield wheat were developed using at least the former technique*. Both techniques have been used to create new cultivars in practically all major crops (both food crops and ornamental crops) that can be sold as Organic. That being said, while you have made an appeal to authority as a scientist, the basis of the argument you've made above is completely non-scientific. That is not to say that your objection is illegitimate, but establishing that it is non-scientific is important (I'll explain why later).
Also, the "mechanisms and pathways which have stood the test of those 2 billion years" result in "catastrophic species loss or damage" all the time . Not all dinosaurs became birds--most just became fossils. Disruptive innovation (to use a less grim and more /.-friendly term) is the norm in the biological sphere, so you might need to revoke your free pass :).
There is no way we can possibly safely understand the full implications of inserting a fish gene into a tomato to improve shelf-life.
As a physicist, you should also know that there is no way we can possibly understand the full implications of moving my coffee mug to the other side of my desk because the n-body problem is unsolvable. We can only approximate the likely effect. Not trying to be glib, just pointing out that not being able to possibly understand the full implications of something is the definition of FUD, not a strong argument against a given action. To be useful in the real world, we have to start talking about probabilities, and how we can gain a working (if not absolute) understanding of the likely consequences of a given action or class of actions.
I hesitate to invoke Hawking style religiosity but I will: Genetic Engineering is "playing God" (no, I'm anything but Christian) when IMHO there is no way we are anywhere near competent yet to exercise such ability. We need to exercise more humility instead. This beautiful planet is the only one we have, or are likely to have for some considerable time to come, and it should be treated with kid gloves.
Funny you should say this, because this is the exact line of reasoning some of my Biologist colleagues use to argue objections to physics experiments being run at the LHC, Fermilab, etc. I try to tell them that while particle accelerators can theoretically produce "micro black holes", it is only true in the literal sense that they can compress mass into a volume smaller than the Schwarzchild radius for that mass. They do not understand that the probability that such a black hole would subsequently swallow the earth is like the probability that two mobsters separated by 100 meters, each emptying 100 drums of ammo at eachother via tommy guns, walk away completely unscathed because every single bullet from mobster A collided in mid-air with a bullet from mobster B. Some effectively think that by using the LHC we're likely to end up creating 'red matter' from the Star Trek reboot. Suffice it to say that quantum physics is a sufficiently advanced field such that otherwise brilliant biologists will not necessarily have even a basic working understanding of the field.
Getting back to biology, the same argument above is just as applicable to modern medicinal technologies, such as the use of synthetic antibiotics, as it is to GMOs. Every time we apply an antibiotic treatment, for example, we roll the dice on the creation of a resistant strain of whatever it is that we're hoping to kill. This has, in fact, already happened many times over, and is becoming a huge problem (e.g., google MRSA, but I suggest that you not do so during or immediately prior to a meal). Using the logic from the first quoted passage, we should stop all medical treatment because the consequences of no treatment are finite and known (or at least backed by a perponderance of empirical data), while the consequences of treatment are unknown, or known to a much, much lower degree of confidence.
The fundamental issue, I think, is that any sufficiently advanced science not only becomes indistinguishable from magic--it becomes scary. For greater values of 'advanced' this becomes true even for scientists in other disciplines. This effect is compounded by the fact that the greater our advancement in each individual scientific discipline, the more difficult it is for any one individual to have an accurate understanding of the current state of multiple scientific disciplines. Physicists were probably the first to experience the full political ramifications of this because, given sufficiently intelligent people, incredible, mind-blowing advancements could be made with relatively primitive technology (Einstein + chalk yields e = mc^2, Schroedinger + cat = quantum physics, etc.). Biology as a field is only now starting to make similar advancements--in large part because, being far more reliant on empirical observation, biology is far more dependent on technological advancement (essentially, our ability to poke, prod, and measure things in increasingly precise and accurate ways determines our ability to advance the biological sciences). The fields of genomics and bioinformatics, for example, would be essentially dead in the water without the massive computing and data storage resources made available by modern technology. Our advances in technology have only recently allowed biologists to make the kind of advances analogous to those physicists were making in the early 20th century.
Biology has, to continue the physics analogy, moved from understanding that matter is composed of atoms, and being able to push them around and rearrange them in rough fashion, to being able to split and fuse atoms in a controlled and deliberate manner (or in a deliberately uncontrolled manner just to see what happens). The field of biology as a science has now advanced beyond the point where the current state of the art is easily and accurately understood by scientists in unrelated disciplines and because of this, it is now scary. Given one result of analogous physics advancements--nuclear weapons--this reaction is definitely not unwarranted.
Despite the above, the reason so many of my colleagues paint GMO objectors as being "pichfork'n'torches" hysterical is that, for the most part, objectors are unwilling to hold an honest dialog. Einstein, for example, was up-front about the fact that his objection to Quantum Mechanics ("God doesn't play dice") was not scientific, and that the only way to mount a legitimate challenge as to the validity QM was to find a scientific basis from which to do so. In the current debate about GMOs, however, there is no evidence of that kind of intellectual integrity**. Anti-GMO groups are willing to say essentially anything, and claim that that they are making scientific arguments. One of the more common tactics is an appeal to authority, where an MD, chemist, agronomist, etc., who does not actually understand the science nevertheless tries (often successfully) to persuade people that scientists or 'scientish' people objecting to GMOs is equivalent to a scientific basis for objecting to GMOs. As I mentioned earlier, non-scientific objections are not automatically invalid, but it is critically important to distinguish them as such if you want an honest, constructive dialog. The problem with this is that a constructive dialog will almost certainly result in a regulatory framework that will control but nevertheless allow GMOs into the market while legitimizing agricultural genetic engineering in the eyes of the public, which is not acceptable under any circumstances to many anti-GMO groups for reasons totally unrelated to science and/or public health.
In my personal experience, there are basically 2 of main reasons groups push faux science to fight GMOs. The first is that they actually do not understand how to evaluate an argument for scientific validity. In my experience, this applies to majority of members of anti-GMO groups if not the leaders themselves. These people are not necessarily unintelligent, they are just not sufficiently versed in science as a discipline to spot misinformation, so they repeat it. Any /. threat on biotechnology yields overwhelming volumes of evidence of this effect in action. The second is that even if they understand the difference, they nevertheless feel it is important enough to fight GMOs for a personal/religious/philosophical/social reason that if lying to the public is the most effective way to do it, they'll lie through their teeth. These are not necessarily immoral people--after speaking at length with some people whom I'd place in this category, I feel they genuinely believe that they are doing the right thing. For example, some activists will use GMOs as a boogie man because they feel it is the most effective way to garner public opposition to biological Intellectual Property laws that they feel pose a legitimate threat to food security. Try "I would like to argue that biological intellectual property laws enforced by WIPO and WTO are a major threat to the economic independence of developing nations" vs. "Ermahgerd, Frankenfoodz!" at a rally.
In the first case above, most biologists I know feel that it is simply not worth the effort to persuade someone when, even if they are successful, they have only gotten the individuals in question to transfer blind faith from one figure to another, not imparted any real understanding. The more cynical/mean-spirited ones feel the ignorance serves as a form of stupidity tax/social darwinism. This is not an abstract and humorous assertion, by the way. It was a dead-serious argument by some when, for example, thousands of people ended up starving in Zamibia because European activists persuaded various governments that GM food aid from the US was effectively poison, and got their local governments to threaten to close access to Euro markets if the GM food aid was accepted. The result was that many people ended up boiling and attempting to eat actually poisonous roots and plants for lack of any other form of sustenance. In the second case there is either a grudging acceptance that science is essentially besides the point, or outright hostility due to the dishonesty. In either case, very few people who actually understand the science see any reason or productive outcome in trying to wade into the debate--hence the perception, which is thus often warranted, that the scientists working on genetic engineering technology dismiss objectors as hysterical idiots.
Apologies for the rant, but you asked for it :).
* The most widespread form of which is inducing haploidy via crossing to a cultivar prone to producing haploid embryos containing only one parent's nuclear genetic material, then inducing chromosome doubling in the resulting plant via mitotic arrest agents to quickly produce new homogenous cultivars (i.e., doubled haploids).
** To be fair, there were few (if any) immediate economic and social ramifications either way in Einstein's case, whereas there is a massive economic incentive to many parties in the GMO debate, which provides both incentives for and grounds for suspicion of ulterior motives.