The big factor is mass. To store energy you need to spin up and down the mass. However to drive in general you want to carry less mass on the vehicle.
Factor #1: A more massive flywheel can store more energy at slower spin rates.
Factor #2: A more massive flywheel is going to be more of a load in general driving.
The optimium point of flywheel mass is going to depend on driving conditions. Really you should have at least 2 interchangeable fly wheels that you physically replace in the vehicle. One flywheel for city driving one for highway driving.
Factor #3: A spinning flywheel is one hell of an energy store. Having a stopped vehichle with a fully spun up flywheel hit could release the spinning flywheel to the detriment of pedestrians in the neighborhood.
Factor #4: Starting from a stop and attempting to corner, left or right, having a spinning flywheel is going to do gyroscopic things to the vehicle.
There are all sorts of tradeoffs and safety considerations here.