While it struck me as baffling too, now that I've thought about it some more, I'm inclined to think that it honestly shouldn't have come as a surprise. 4K blu-rays entered the race late, stumbled out of the gate, and found that they were not only entering a market in rapid decline, they were doing so without the benefit of the factors that allowed their predecessor to enjoy a middling level of success.
The fact that the Xbox S supports 4K blu-rays does nothing to change the situation at large, since it's a mid-cycle console upgrade that flew by the majority of the public without them even realizing it had happened. There's virtually no chance of it changing the situation that drove Sony's decision.
Mind you, this is much to my chagrin, since I'm still in the habit of buying physical media and making high quality rips of it so that I can control where and how I view it, but it's pretty clear that Sony has looked at the sales numbers for 4K discs up to this point, seen that they've failed to turn the tide of declining sales, and have decided that it's not worth the couple of extra bucks per console to support a format that will never manage to gain traction in the market at large.
Just to break things down a bit more, since I realize I said a lot of things without explaining what I meant...
Even though blu-ray won the format war over HD-DVD, blu-ray sales have been declining for about five years, with the decline hitting double-digit percentages for the last three years straight. It's a format in falling-off-a-cliff levels of decline, despite no physical successor displacing it. DVD sales are down by a similar amount. On the flipside, streaming is up. WAY up. The writing is on the wall for physical media, and has been for some time, whether we like it or not.
This is happening, despite the fact that blu-ray had every opportunity to succeed. Getting people to buy an expensive new TV would typically be a major hurdle, but the falling price of HDTVs around the time that blu-rays launched (coupled with the fact that massive, flatscreen HDTVs were a compelling upgrade over the bulky, small CRTs that everyone had at that point) helped drive an incredible adoption rate that far outpaced the typical upgrade cycle.
In contrast, while 4K blu-rays have to cross that same hurdle of getting people to buy new TVs, we haven't seen 4K TV adoption move anywhere close to the same speed that we saw with HDTVs (i.e. it's on par with the typical replacement cycle). 4K TVs are proving to not be a compelling upgrade for most people over existing HDTVs, presumably because the benefits of UHD over HD are lost on most people.
Late Arrival and Competition
Moreover, YouTube, Netflix, and other streaming services have been providing 4K content (or higher) to the people who wanted it for years at this point, and they did so without requiring big ticket purchases of their users. The first 4K blu-ray discs didn't start shipping until March of this year, so the format was both late to the game and priced higher than its competition, meaning that it's, unsurprisingly, fared quite poorly in the market.
On the other hand, we've seen Sony work on making PlayStation Vue a more compelling service. When the Olympics were ongoing, I saw numerous mentions of it as a viable alternative to cable subscriptions and services like Sling TV. And this is in addition to Sony's ever-expanding, existing library of digitally distributed games and movies available in the PlayStation Store.
All of which is to say, between the market and Sony's other moves, it shouldn't have come as a surprise to us that this would happen, even though we might wish otherwise.