I was a ballot observer last November in WA for a morning. The head of the elections office showed me around, and I was pretty impressed with their method. Ballots are counted at every step of the way from the ballot box and meticulously documented.
In addition, there is added security by using two envelopes to hold the ballot. The outside enveloped has your identification signature on it, which is compared to your signature on file from your driver's license or your voter registration by a human being. I've personally received a call from the office because I started using a different signature after college, and was told to come down to the office to verify my signature before it would be counted. The outside envelope also contains information about how many ballots you've been issued (if you asked to invalidate a previous ballot that hasn't been verified and counted yet. Additionally, if you were coerced, you could cancel your ballot, and ask for a new one, because the first has an identifying number on it.
After verification, The outer envelope is opened, and your ballot secured in the inner envelope is removed, and placed with all other ballots to be counted, ensuring anonymity.
You might say that the wrong people in charge of your ballot could throw an election, but that is true whether it is paper or electronic. I consider paper to be more secure, because it requires a considerable effort to destroy or modify, and recounts can be done by different staff and/or with public oversight. Electronic election fraud could be committed by changing an entry in a database.
