Years ago when I finally went away from my phone/PDA, this was before large tablets were mainstream so I had an basic phone/n810 at the time, setup I found what you described to be the case. Market was a bit different of course and so I went with an entry level Blackberry.
Time went on and as that Blackberry got older, along with the whole platform being crushed by Android/iOS, I wanted to upgrade. I went to my carrier and at 1st was going to again go with a Blackberry but was told that to upgrade to a newer model I would also have to upgrade my plan. And the reason for the plan upgrade was I "needed more data." But the kicker was that I was not getting anything that would use any more data. Doing anything really internet related on the oldschool Blackberry phones that had physical keyboards and tiny screens was always a last resort.
So I started shopping and found also that really the contract less plans mostly were the better value in terms of TCO even with the drawbacks of a lesser network, having to buy your own phone outright, and less options when buying said phone. What was driving this market I had to ask myself.
Now I've not done any deep analysis but it seems that the carriers have captured the market in two ways. Keeping TCO higher by putting the phone cost into plans thus spreading out the cost for people. But the real thing is the upgrade treadmill. The very nature of smartphones makes them highly susceptible to wear and tear, if not outright fatal damage, aging electronics/batteries, and of course planned obsolescence.
There is a market for people who don't need the latest phone, if a smartphone at all, but it is not big to begin with and nobody is marketing to it. It is much more profitable to keep the people who would shift over to a cheaper TCO for their own needs in the upgrade treadmill market.