Something very similar has happened in my town - a paved bicycle rail trail was proposed, which is currently a decommissioned "abandoned" railroad. The rail trail real estate itself is clearly unused. Anyone owning property adjacent to the existing "railroad" has a legal leg to stand on for disputing said bicycle rail trail in court. There are a handful of wealthy lawyers who are property owners adjacent to the proposed bicycle trail in each town the rail trail cuts through. These people thoroughly detest the idea of having smelly, loud, potty-mouthed cyclists constantly near their property (I can't say that I don't blame them) and continue to use their expertise of the legal system to dispute the proposed rail trail in court. This has been going on since the mid 90's, and the rail trail still does not exist. The total length of the proposed bicycle rail trail is under 10 miles. If it is this difficult to get a short bicycle path created out of an existing unused railroad, I seriously doubt fast railroads like the ones currently in Europe and Japan will ever appear in the US.
Besides, look at the Acela. It's the fastest train in the US, but it can only travel at it's top speed for a very short distance near Attleborough, MA where there are no street crossings. When it's not near Attleborough, it lumbers along like some amusement park railroad. What are the options for solving this? Have everyone and their brother-in-law get big bucks construction contracts to modify the crossings into bridges?