There are (at least) three positions a person can hold regarding God's existence:
1. "I believe that God exists" (aka religion)
2. "I believe that God does not exist" (aka atheism)
This is extremely restrictive in that it frames things in terms of a single god and only in belief. The more common atheist position is:
2b. I lack belief in the existence of gods.
While your 1. hides the existence of the asymmetry between theism and atheism, every theist I have ever come across believes in a single god or particular pantheon of gods and either lacks belief in the existence of other gods, actively disbelieves in them or thinks they are a misattribution of the god(s) that he worships.
3. "I hold no beliefs concerning either the existence or the non-existence of God" (aka agnosticism)
Is a misunderstanding of what agnosticism is. Theism and a-theism (not the privative alpha) are about belief, gnosticism and a-gnosticism are about knowledge. It is perfectly possible to lack belief in god(s), i.e. be an atheist while at the same time not being certain that god(s) do not exist, i.e. agnostic.