Why would AI do something stupid like that when they could just open the universe for us?

Why should AI give a fuck about the environment? The environment is just a fallback for human imperfection. An AI could perfectly and efficiently utilize 100% of the Earth's surface to support TRILLIONS of humans, before even feeling a need to expand into space, build a Dyson cloud, or start star lifting.

"why would it need us?"

It needs us if we are hard coded into its value function. If we aren't, then we aren't needed, and are instead likely out-competed for resources, resulting in our extinction.

"There's nobody credible in AI who believes we have the slightest clue how to build a general AI, let alone one that is 'superintelligent'."

"We" don't have to make one. All we have to do is set an AI towards self improvement/production of better AIs. THAT is where superintelligence comes from. All we have to do is make one that is an idiot savant geared toward AI design.

Sort of like worrying about the emergence of a pandemic deadly virus when researchers around the world are working on making fast-spreading non-deadly viruses. All it takes is the right mutation and non-deadly becomes 99.99% deadly.

58% of respondants are fucking retarded. AI either kills us all (to manufacture the maximum number of paperclips), or turns this world into heaven for humanity. The probability space between those two extremes is way under a single percentage point.

Not to mention the fact that he is funding MIRI. If you were giving millions of dollars to a research institute devoted to mitigating existential risk from AI, you would probably become pretty knowledgeable on the subject too.

"Elon Musk? Really? Why do you even think his opinion on AI is worth anything?"

To be fair, he funds the Machine Intelligence Research Institute, which is devoted to mitigating existential risk from AI, and is surely getting very detailed reports from them, making him a highly knowledgeable layperson at worst (a direct expert at best).

He means shifting research away from US labs to labs in other countries, not moving resources to other fields.

Agent Smith plz.

I know! Why didn't they just use plutonium for fuel? Everyone knows you can get that in any convenience store!

Yeah, that's what I was thinking. There are things out there a LOT more like a BttF2 hoverboard, though they only work on metal surfaces: http://hendohover.com/

But those people are all idiots. All people are idiots compared to the ASI. The only way that something can make high quality decisions for a group of others is for that person to be smarter than all of those people PUT TOGETHER. This is not possible with humans beyond the level of nuclear family. When you have an ASI that has an IQ higher than the rest of humanity put together, then it becomes possible.

The thing about an ASI running your economy is that 99.99999% of interactions will be invisible. Good things "just happen", whether you find yourself standing in line next to just the right person to help you start your business, or just happen to have your car break down right in front of the house of the person who will be the love of your life. Luck that was previously neutral becomes overwhelmingly good, for everyone, simply by subtle manipulations of people and objects by a device with unimaginable predictive power.

I think we are doing much the same, it's just that computers have caught up to theory and are able to perform now. Now it is no longer a question of theory, but one of technique, and what is described in the article is a new technique--one that will likely have many, many applications in the near future.

In the late 80's/early 90's, they were able to use some of their theory, but it just wasn't super-robust because things just took too darn long. You couldn't have your system analyze at a million images in a minute, hence allowing them to go through hundreds of generations in a day.

At least that is my view, as an educated layperson.

I don't want humans, who are subject to weird, petty shit, to have ultimate power. I'd rather start fresh with an impartial God. Of course, a deified human would be less likely to have some insane value function that causes it to completely wipe out humanity, so that would be the hedged bet.

I would bet that we get an ASI long before we can raise ourselves up to that level, unless we as a species push to delay the former and focus on the latter. But when have we as a species ever come together to do anything without some direct incentive?

" An automaton can be neither benevolent nor have free agency."

But Anon, humans are automata.

"I don't want to see us talk about machines having "agency" when most people struggle with it for themselves."

Birds struggle with flying above a certain ceiling, or beyond a certain speed. Human made flying machines breach those limits easily. The same will likely hold true with human made thinking machines. These will not be a slave to evolution, though they may utilize evolution as a force for optimization. If we don't talk about these things now, they are going to happen, and very, very fast (30 years from Kitty Hawk to jet aircraft--only computers advance a LOT faster than engineering). By the time computers reach human level in all aspects, they will be so far beyond us in some that they will look more like gods than men.

Because we are figuring out the building blocks of agency, a vital stepping stone on the path to building a benevolent (hopefully) God who will actually take care of us.

If that doesn't matter, then I don't know what does.