Note: this post is from a UK perspective, things may vary a little arround the world.
By the 1980s, we had developed mathematics and modems that could use the same lines to get up to about 33.6kbps at 3,429 baud.
And then things more or less stopped there. There was one more marginal speed increase (56K) but they had pretty much hit fundamental limits of the phone system. Pushing speeds further required bypassing parts of the phone network.
ISDN BRI delivered slightly better speeds in the 90s but the way it was priced (if you wanted a 128k connection you had to pay for two phone calls in addition to the ISDN line itself costing more than twice what an analog phone line did, AIUI most unmetered dialup packages allowed single channel ISDN but not dual channel ISDN) made it an expensive option. ADSL turned up in the early 2000s but again it was initially expensive.
ADSL gradually improved through the 00s first with the providers getting more confident and taking the artificial limits off and then by the providers moving to ADSL2. However while speeds improved so did the gap between the haves and the have nots. Those close to the phone exchange could get 20mpbs, those stuck a long way from the exchange got less than 1mbps and we have pretty much hit the limit of what phone cables can carry over long distances even with advanced modulation techniques.
In any case, odds are, whatever we put in the ground today, in 20 years we'll be able to do more with it than we can today.
The problems with mixed fiber/dsl systems don't really have anything to do with the fiber that is being put in the ground.
1: it still relies on that old phone wiring for the last hop. There are a few tricks we can pull but we have pretty much hit the limits of what those cables can carry over those distances. You still have the "cable length lottery" except now it's distance from the point of fiber to copper transition to the house rather than distance from the phone exchange to your house.
2: having all that infrastructure spread out like that makes it very difficult to do incremental upgrades. When ADSL was introduced they could start by putting one DSLAM in a phone exchange and patching the subscribers to it, when one DSLAM filled up they could add another. It didn't matter that only a few percent of customers were taking DSL intitially because the phone exchange was large. On the other hand there were places in the UK that had their POTs and ISDN delivered over an early fiber to the cabinet system and these were among the last to get ADSL because it wasn't worth putting a DSLAM in a cabinet for a handful of subscribers. So even if there was a system that could get a slight improvement over the current VDSL gear rolling it out would be very expensive.
The only real way to substantially improve a "partial fiber" system (fiber to the cabinet, fiber to the distribution point etc) is to push the fiber closer to the subscriber but each time you do that your infrastructure ends up even more spread out. Eventually you get to the point that you may as well just take the fiber all the way.
On the other hand with a fiber to the home system all you have to upgrade to deliver faster speeds is the consumer premises equipment and the exchange equipment. All the outdoor infrastructure can remain the same. Plus current fiber to the home equipment has a much wider margin over current needs than the VDSL that is being deployed in current fiber to the cabient system.
Deploying fiber to the cabinet now means in a few years time either internet speeds will stagnate again or a fiber to the home project will be needed anyway making the fiber to the cabinet equiment redundant.