Or they are willing to let customers go. If you only use it for free two day shipping, then maybe $99 is too much. Then you stop using the service. And Amazon profits go up.
Here is why i originally bought Amazon Prime: because it was taking a week to get an order. Not because shipping took a week, but because packing often took a few days. Prime forces Amazon to prioritize my orders. That means costs beyond what they would be paying if they could ship 5-8 days. I can imagine these costs to be increasing over time, not paying the shipper mind you, but paying enough staff to pack so that my order leaves the warehouse in time for the shipper to get it to me in two days. All for $100.
If I order a $20 widget a week, that is around $2 per package shipping, around $1000 revenue, and probably zero profit. Yes, amazon is shaking in it's boots because people who only use shipping are going to cancel.
OTOH, if I didn't have amazon prime I would be more likely to shop elsewhere.