I believe the difference is that by the time you're facing a breathalyzer--which as you point out, there are penalties for refusing--you've committed some sort of violation. At that point, you're interacting with the police and if they have reason to believe you are under the influence--either due to the previously cited violation or via observable signs (smell of alcohol, slurring of words, etc.)--they can begin the series of tests to confirm their suspicions.
In this case, you're just pulled over and and checked for no valid reason other than everyone is being checked.
Is now when we drag out the "If you haven't done anything wrong, what do you have to worry about" line?