I would argue that most of the scientists you hear arguing for atheism have absolutely no clue what they are talking about, because they assume that if it exists in any way, it can be reached scientifically, and that anything that cannot be reached scientifically, cannot exist (that combined with their reluctance to trust the authority of anyone or anything they can't understand tends to lead them to atheism). That doesn't follow.
It's true that it doesn't follow, but an entity that does not in any way interact with reality may as well not exist for all I care. If on the other hand it does interact with reality, that interaction should be observable - and hence part of scientific inquiry. So while it's strictly speaking true that non-existence does not follow from non-observability, you seem to be attacking a strawman argument by shifting from practical non-existence (i.e. having the same effect as not existing at all) to actual non-existence (i.e. the claimed "cannot exist").
Mind you, anything that occurs in the natural world does (so magic is right out), but you cannot rule out the possibility of supernatural beings existing.
By your logic we cannot rule out the existence of magic either. It may simply have effects outside of our natural world.