A little background (good question): To the best of our knowledge, there's no such thing as gravity "on" something; all gravity is *between* things. Think of it like magnetism (in fact, if you change the constants and swap charge and mass, the formulae for computing magnetic attraction and gravitational attraction are the same). When the Earth's gravity imparts momentum to an object (an apple you drop, say), the apple's gravity imparts the same momentum on the Earth. Of course, since momentum is mass times velocity, and the Earth masses ludicrously more than an apple, the delta-V of the Earth is basically imperceptible. Momentum is still conserved, though.
Now, as far as a space drive goes... we can't use gravity as a drive right now, because it always just pulls towards nearby massive objects (and because we can't control it in any way). We can accelerate using gravity - you've probably heard how some space probes would "slingshot" around massive planets to gain a lot of speed on a different vector - but in order to do that we first need an acceleration that we create ourselves, so that we don't just fall straight down the gravity well.
So yes, gravity imparts momentum (to both spacecraft and the planets they slingshot around) without *itself* involving a high-momentum exhaust... but only because the spacecraft already had a lot of momentum in the correct direction for the maneuver. Getting *that* momentum has, so far, always required an exhaust.
There are other options for generating thrust in space - light drives (the "exhaust" is just massless photons) and solar sails (where the high-momentum particles come from something else, like a star), for example - but neither are currently practical. Of course, even if the EmDrive happens to really work (which the experiments support but have definitely not yet proven) it isn't yet practical either. NASA has tested a *lot* of experimental drive types. However, at this time, all of the ones that have actually flown are reaction drives (throw something out the back of the ship, get an equal and opposite reaction forward). That may change at some point in the future, though.