Unfortunately, even though you wouldn't have to supply propellant, you would still need to supply the energy to accelerate it.
Actually you don't. Any reactionless thruster can be turned into a free energy machine. If force-produced is directly proportional to energy-input (eg, the device in TFA gets 31uN/W) regardless of position in the universe and direction of thrust (**), then the change of velocity per unit time is also proportional to energy-input. Linear. But the kinetic energy available is proportional to the square of total velocity. There's a cross over velocity where the increase in kinetic energy is greater than the energy input necessary to produce that change in velocity. Stick a set of reactionless thrusters on a flywheel, hook the hub to a generator, spin it up until the tip-velocity passes the cross-over, and from then on it's a free energy machine. So the potato would have one set of EMdrives powering a generator, and another set for propulsion powered by the generator.
At 31 microNewtons/Watt, the required tip velocity is over 30km/s, a wee bit past material limits. But as the "Q" increases in the EMdrive (predicted for the use superconducting cavities) the cross over velocity drops below 100m/s, well within the tip-velocity of a generator flywheel.
(** An anti-gravity drive that could only act in the direction of existing gravity, and lost efficiency moving out of a gravity well or at speed within a gravity well, wouldn't be a free energy machine. And the inventors of the EMdrive have indeed created some ad-hoc explanation for why theirs loses power at speed (to which every critic screams back "relative to what!") and can thus magically only be used to hover.)