If the new city people call the cops and you're not doing anything that's against county laws, then the cops can't do anything.
This is exactly the situation and the sheriff's department usually follows up on any calls they get and sometimes you get a condescending deputy regardless of the law (sometimes because the neighbor told them something that isn't true or out of proportion). At one point I had talked to an attorney and he recommended filing a harassment complaint but fortunately those neighbors moved away before that became necessary.
But my main point is those of us who live in the country have an influx of city dwellers who have dreams of Green Acres but when they find that country living isn't the same as what they imagined some of them start harassing their neighbors until they get their way. I recommend anyone who wants to live in the country maybe rent a place, if they're serious, and live out there for a few months and see if it fits them before making a serious commitment to an environment they may not feel comfortable in, otherwise they might want to stay in their suburban HOA.