Currently the ISPs market "Up to 50mb!" but thats only if no one else out of your remote is currently online.
If you have 50Mbps over phone lines, you have VDSL2 and VDSL2 remotes typically have at least 20Mbps of available upstream capacity per port so if everyone on the same remote has 50Mbps service, about 40% of people connected to it can simultaneously use their service at full speed before the remote actually becomes a choking point. This part of the service is something the ISP has full direct control and visibility into. Even ancient ADSL1 DSLAMs had the ability to probe lines for service quality monitoring/provisioning purposes, everyone knows performance on xDSL depends on line quality and that part of the service has absolutely nothing to do with network neutrality.
Where things become far less predictable is when the traffic leaves the ISP's middle-mile infrastructure, interconnects with peers and transit providers, internal hops across those external parties the ISP has absolutely no visibility into or power to do anything about, interconnect between those third-parties and others beyond, the far-end interconnects between those third-parties' third-parties, the far-end network, etc.
If you want network neutrality to start defining some degree of end-to-end performance guarantees (unless further limited by technical limitations such as maximum sustainable line sync), the whole internet would be affected; not only the first-mile operators.