It's an interesting question; where should we draw the line? Obviously, if your embryo/fetus is destined for cystic fibrosis, we would all cheer for a genetic intervention. But if you just think it would be cool if you kid would glow in the dark, it may be quite reasonable for the rest of us to resist allowing those "imported" genes into our collective gene pool.
I think a key concept to consider in all genetic modifications is the difference between vertical gene transfer and horizontal gene transfer. The former is manipulation of genes within the same family tree, whereas the latter involves importation of exogenous genes from other species.
If you ask me where I would draw the line, in terms of legislation, this is it: endogenous (vertical) vs. exogenous (horizontal) gene transfer. Do whatever you want within the human genome, but don't import any genes from any other genome.