Voice has to be given priority to keep the latency within acceptable limits (otherwise you get an effect like satellite delay).
The flight time from the phone to the tower is fixed (based on speed of light), but the time when your phone is next allowed to transmit (i.e. when its turn comes) can be altered, as can the back-haul (i.e. the link from the tower to the internet or tower to voice gateway).
I suspect (but have no proof) that data is deliberately retarded and jittered (e.g. volatile ping time) to make VoIP very poor quality over the cellular network.
I see no technical reason why I shouldn't be able to buy high-priority traffic (i.e. at a premium) and use it for whatever I want - be that VoIP, gaming, or just because I want that download now!
Its all about maximising profit per user, not providing the best service.