I'm trying to figure out how forcing someone to provide a service to another person (for whatever reason) does not have an affect on the person PROVDING the service.
For the cake buyers: their initial state is cakeless.
When they are refused a cake: their final state is cakeless.
There has been no change to their state of well being or a violation of any liberty. They are in the exact same point as where they started and are not restricted from making their own cake or finding another baker. They may be slightly upset over having their cake order refused but there is no constitutional protection from having your feelings hurt.
For the baker: Initial state is he is living his lives according to their beliefs
After being forced to cater a gay wedding: not only has he violated his beliefs but he's had to sacrifice his resources (time and product) to do so.
Only 1 party in this transaction has had any major change in their status and that is the baker.
If you disagree with this baker the proper option is to open up a competing bakery which does provide services to gay weddings and let the market decide. In reality, in one bakery case the owner was told that he would have to provide services for all gay weddings in the future or face up to 12 months in prison and another closed his shop and started working out of his home because of the harassment from organized gay rights groups and death threats to his children.