No regulations, no permits, no artificial barriers to entry. Open competition. Cool.
I want to lay my own fiber in your neighborhood. So I bring my backhoes and dig up your street and lawn. Barbie down the street wants to lay fiber too, so she gets her backhoes. Perhaps you can see that unregulated open competition for infrastructure would be a big mess.
I have libertarianish views, and I hear what you're saying, but what you call the problem is not really the problem. The problem is trying to privatize infrastructure we all share.
The most logical entity to own infrastructure is The People. Call me a socialist, communist, whatever, but that's how it is.
But people don't like government, so they "privatize", which is to say, hand a monopoly to private hands.
So your choice. Regulated public ownership. Regulated localized monopolies. Unregulated libertarian fantasy of every american with a dream driving backhoes through your yard.