No, liquid fuels don't require a lot of NEW infrastructure to store and transfer.
A lot of the infrastructure installation costs have been paid already, but that doesn't mean the tanks, piping and gobs of safety equipment isn't there.
At the dawn of the automobile industry there was no infrastructure setup for gas cars, everything was setup for horses. That quickly changed as stables went out of business and gas stations popped up. All that change wasn't free.
All in all the transition to electric vehicles would be a lot cheaper and less disruptive than the one from horses to gas cars. There are still a good number of similarities between the two technologies, if only in form. And a current gas station could fully transition to electric by replacing their underground storage tank with an underground battery/capacitor bank, no need to completely redesign their business. The first big benefit is the lack of gasoline spills/leaks to worry about, cutting down on the need for a fluid barrier underground.