Sounds like you need JACK, the linux audio system meant more directly for production. A low-latency tuned kernel could also help.

This is a great reason to bring in SmartCards + anonymizing Digital Signatures into the realm (IIRR Belgium has an interesting voting scheme around this).

Paper trails are a bunch of BS if you ask me. Why would anyone trust paper more than cryptography?

You need to be able to have multiple backends in different locations simultaneously being posted vote result envelopes.. as well as the individual machines containing their own logs.
At the end of the day you'll have instant voter readout after you have the various machines correlate votes and make sure no duplicates/inconsistencies are made by the entitys. ... this is the way to do it without the waiting involved with paper-based ballots.

Now.. OpenSource is a strong req end-to-end on the machines so that no invisible bogusity can be implanted. Why would you trust a black box to count your votes?