American passenger rail is dreadful largely because rail can't compete with air over the distances Americans want to travel. There are some plausible high speed rail routes, like Boston to Washington and maybe LA to San Francisco, but rail can't compete on speed with the longer routes people routinely want to take. Air travel has a substantial advantage over rail for travel from the East Coast to the Midwest, and utterly crushes it for travel from coast to coast.
At the same time, American freight rail is substantially faster and more efficient than European freight rail. That isn't a coincidence, either. On both continents, there's a lot of shared rail that has to serve both passengers and freight. In Europe, passengers take first priority, and in the US freight takes first priority. Passenger service in the US supposedly got much better during the worst of the Great Recession because there was enough less freight that it made a big difference for passenger trains' speed and reliability.