The minute the locked tubs leave your sight, you are relying on an army of other people to make sure they aren't unlocked while you aren't looking and the ballots switched, so no, you can't verify the paper trail when you vote.
With an end-to-end verifiable paper ballot protocol like punchscan and integrity, however, you (individually, without relying on an army of eyeballs to preserve chain-of-custody) can verify that the vote count is accurate because throughout the election (from the printing of the ballots to the scanning of the ballots to the final vote tallies) enough information is made public about the ballots and how they are marked that it is impossible for the election authority to steal more than a few votes without being caught. The probability of one vote being stolen without being detected is 1/2, 2 votes 1/4, 3 votes 1/8, and so on. At the same time, the information revealed is not enough to determine how any individual person voted, so anonymity is preserved.
At the heart of such protocols is the concept of a cryptographic commitment. Suppose you and I want to flip a coin fairly via an email exchange. If we were face-to-face, you would call heads or tails, I would then flip the coin, and we would see who won the toss.
How do you prevent cheating in a similar exchange over email?
The key is that I flip the coin first, generate a commitment and email it to you, then you call heads or tails, and then I reveal the key that unlocks the commitment, whereupon we both know who won the toss.
How do I generate a commitment that I can't modify later? Suppose I flip the coin and it comes up heads. I then generate a 128-bit random number, concatenate "heads" to it, and calculate the SHA256 hash of that string. I send you the hash. You call heads or tails. I then send you the 128 bit random number I used, and tell you it was heads. To check me, you take the random number, concatenate heads to it, and calculate the SHA256 hash. If it matches what I originally sent you, then you know I didn't cheat. If it doesn't, then you know I tried to pull a fast one.
How do you build a paper ballot election protocol out of that? That's a longer story. If you are interested, I wrote up my explanation of it here: http://seedsofgenerality.blogspot.com/2010/09/secure-voting-protocols.html
Now I grant you that not very many people will take the time to understand how a complicated protocol like that works. But it would only take a small minority of people to vet such a scheme so that the wider population would have confidence in it. Just as it only takes a small minority of people to understand how RSA works for the rest of us to use ssh with confidence.