It's not only infrastructure. Not even starting at wages for workers and other recurring costs, ISP's have to pay each other to buy bandwidth from them. Only the tier 1 ISP's can get away with peering without extra costs.
On top of that, their payment model isn't $0.xx/GB of transfer, it's $xxxx per Gbps of bandwidth. If ISP buys too much bandwidth, it means it will sit there without being used, or it may be used in peak times and be just sitting there the rest ~20 hours of day.
Also, it requires there to be actual connectivity available - for example, my country as a whole has something like 30Gbps in/out. Still they're selling 100Mbps for customers to use. It's perfectly sure that if everyone would use that 100Mbps at once to download content off the country, it would not be enough. However, it works out because home users rarely need that kind of bandwidth 24/7.
If you want dedicated bandwidth that is guaranteed, then buy it. Just be willing to pay over $1000 a month for your 100Mbps line. This isn't new to anyone - it's the same in server hosting world too, but there it's more clearly marked if the bandwidth is shared or dedicated as it matters more and some people actually have real need and are willing to pay up to that $1000 a month for it. With home users, no one would do that.