Comment Not that scary (Score 5, Insightful) 85
This isn't that scary when you realize what's happening and why they are more persuasive.
They are using tactics that are known to be effective, but that most people have a hard time actually doing because, emotionally, it's very difficult. Those who can do it end up being just as persuasive, if not more persuasive, as an AI - IF they have equivalent knowledge of the other person.
It was Pascal who pointed out that to persuade people you should first acknowledge their points of view and lead them to discover the counter-arguments themselves rather than imposing your own.
This is difficult emotionally, at least for me, because when talking to fascists, for example, it feels morally wrong to acknowledge their point of view. But if you can state their point of view in such a way that they identify with it ("yes, actually, that's how I feel and think") - maybe even state their beliefs better than they are able to themselves, with intimate knowledge of their concerns, it immediately builds trust with them and takes their defenses down a notch.
They may allow you to then help them discover the counter argument themselves, with some careful guidance.
This takes skill and an emotional resolve that few possess when discussing things that feel morally important.