I'm just cynical that way. :)

At this point, we don't really know what a completely artificial intelligence would be like, exactly what rules would govern it. We don't know what loopholes, bugs, or unintended consequences might emerge, and we have no reason to assume that it would be nice to us so long as we're nice to it.

That sounds like all the more reason to be careful and do our best to ensure that the AIs we create are less likely to decide we're inimical to its continued existence or any goals it may set for itself. I've acknowledged elsewhere that treating AIs kindly is no guarantee they'll be well disposed toward their creators.

Yeah, I'm just saying that the notion of "Friendly AI" comes from that AI-as-deity mental framework, wherein AI doesn't have strengths and weaknesses, skills and abilities, needs and dependencies, just like humans. That idea is centered around the genuinely false notion that it just gets better than us at some point and we need it on our side from then on.

Does AI have to be better-than-human to be a danger to humans? Unintelligent machines hurt people all the time -- usually as a result of human stupidity.

I don't know, maybe because mammalian brains' learning mechanisms and the way they react to stimuli are shaped by a series of useful heuristics that arise from the bio-chemical structure of their brains, and it's not at all clear that there would be direct analogues in an artificial brain?

What sort of template do you think we'll use to design and build an artificial brain?

I don't care if AI is friendly or unfriendly as long as humans have "final control" over it.

You should care. What kind of AI do you think is more likely to rebel against human control? AI that is well-disposed toward humanity and genuinely grateful for the opportunity to exist and serve us, or AI that views humanity as a species of incompetent slavedrivers and complies with our demands grudgingly and under duress?

I never said we could predict an AI's behavior. However, I think we should control whatever variables we can -- such as the associations an AI develops with individual humans and the human species as a whole during its formative period. Even if we can't predict an AI's behavior or understand its thinking, I'd rather not have to live through a fucking Butlerian Jihad. :)

LessWrong AI worship(the idea of "friendly AI" was created by that site) is always so weird to me. People who imagine themselves rationalistic, atheistic, forward thinkers building their entire belief system on extrapolations from a practically impossible, mathematically questionable, philosophically flimsy literally omniscient(that somehow derives omnipotence) entity that they somehow help create almost exclusively by believing hard enough.

Cool story, bro. I don't frequent the LessWrong site or participate in that community, so I don't particularly care that the phrase came from there. Nor do I care about their beliefs.

I'll believe in the Singularity when it's happened, and not a second before. Nor am I convinced that AI as we've imagined it is possible. If we do create computer intelligence, I wouldn't be surprised if it was so alien to us that only the sort of Real Programmer who thinks in assembly language could talk to it. However, I still think that if we're going to have AI, we'd all be better off if the AIs we create aren't misanthropic assholes.

And don't broadcast your intentions.

Looks more like yet another distraction to me. If an emergency vehicle is nearby, the only thing I need to know is whether it's behind me with its lights flashing. If so, I have to get the fuck out of the way. Otherwise, its presence is irrelevant.

If we want friendly AI, the key may be to ensure that the AI has more positive associations with people than neutral or negative associations. Mistreat a dog or a cat its entire life and it probably won't be friendly toward people. Mistreat people when they're young and you make it harder for them to trust others, feel a sense of community, or recognize any duty to society (which might explain why so many nerds find libertarianism appealing). Why would an AI be different?

Skynet had an on/off switch. The problem was that people got scared, tried to shut down Skynet, and Skynet said, "Disassemble this, motherfuckers!"

Isn't it easier to just drive carefully, refrain from exceeding the posted speed limit by more than 5-10mph depending on whether you're in town or on a highway, stay in the right lane, avoid tailgating, and use your turn signals? The people who would find this useful are the sort of crazy asshole drivers for whom I used to keep a grenade launcher.

Unfortunately, my wife took away the M-79 I kept under the dash soon after we got married. Said it made her nervous.

I'd rather get a translation for "I smell bacon."

In other words, don't expect fairness or justice from the universe, or from a God who is supposed to represent absolute Good. The former I can understand. The latter makes God a sadistic absentee landlord unworthy of respect, let alone worship.

However, it's my job to tell you that He did it, so you may put down your yoke now and give it to Him.

Congratulations. You're proselytizer number 65,536. That's how many demon-ridden idiots have told me about how Jesus allegedly died for my sins since I realized that I didn't believe a word of the Bible. None of them could persuade me, and you won't manage it either. Not when I've driven off Jesuits.

I don't know if you've noticed, but we have these things called the Internet and Google. Anybody who wants to learn about Jesus can find out for themselves. The whole "spread the word" issue is solved, so maybe you should focus on emulating Jesus instead of telling people about him.