Possibly I don't know what is meant by perpetual motion.
Well actually I think the term "perpetual motion" isn't particularly helpful. You're right that it's easy to imagine systems that undergo a certain periodic motion without end, as long as they are not disturbed (no energy extracted). Some people call that 'perpetual motion'. Other people might reserve the term 'perpetual motion' for discussions of non-ideal systems (i.e. real systems in our universe), which are subject to incidental effects (like friction). For real systems, there are going to be additional channels of interaction that allow energy to move into other systems, and thermodynamics (entropy wants to increase) thus guarantees that these channels will be used, preventing cyclic/periodic motion from being endless. (One can imagine gas atoms randomly moving around until the end of time, normally that random motion wouldn't be called 'perpetual motion'.)
However, if left alone, my understanding... is that it will keep orbiting forever assuming the orbit is stable and is left alone.
In Newtonian physics, this is true: the orbit will continue forever. (An unremarkable example of perpetual motion. Unfortunately only theoretical since our universe doesn't strictly obey Newtonian physics.) In relativity theory, the orbiting bodies will emit gravitational waves, which means they are slowly radiating away energy. The orbit will decay and the system will eventually end up in a minimum-energy state where there is no orbit/motion. (The ambient random energy of space-time will have been increased: hence entropy increased.)
I believe there are solutions to the equations of general relavitity which include time-oscillatory behavior but do not emit gravitational waves. (Wikipedia says that an ideal spherically-symmetric pulsating mass should not.) But these kinds of ceaseless motion are different than what is being proposed in TFA.
When I said this looked like a perpetual motion machine and that was nothing that extraordinary that is what I was referring to... stable systems that don't degrade assuming you leave them alone.
Right. And if that's all that they were talking about, it wouldn't be that impressive. E.g. one can imagine putting an ion in a magnetic trap, giving it a small push, and watching it orbit around through the trap without end. Call it perpetual motion if you like. (In reality it will be radiating electromagnetic energy and slowing down imperceptibly as a function of time.)
The novel thing about these 'time crystals' is that they would exhibit motion and yet be in a minimum energy state. There would be no way to extract energy from their motion. This also means that the motion would be truly perpetual in the sense that no incidental process could cause the motion to stop, since there is no excess energy to be radiated away. This is the novelty of the new states (assuming they actually exist).