world in conflict unlimited reinforcement points

As Stuart says, “you can’t fetch the coffee if you’re dead.” Stuart thinks this design architecture, if implemented, would be a big step forward towards reliably beneficial AI.įor instance, a machine built on these principles would be happy to be turned off if that’s what its owner thought was best, while one built on the standard model should resist being turned off because being deactivated prevents it from achieving its goal.

And it figures out what we want by observing our behaviour.

But the system isn’t sure what we want.

The AI system’s objective is to achieve what humans want.

According to him we need to redesign AI around 3 principles: Stuart isn’t just dissatisfied with the current model though, he has a specific solution. If we’re ever to hand over much of the economy to thinking machines, we can’t count on ourselves correctly saying exactly what we want the AI to do every time. This ‘alignment’ problem will get more and more severe as machine learning is embedded in more and more places: recommending us news, operating power grids, deciding prison sentences, doing surgery, and fighting wars. Like King Midas, who asked to be able to turn everything into gold but ended up unable to eat, we get too much of what we’ve asked for. This isn’t something we wanted, but it helped achieve the algorithm’s objective: maximise viewing time. Stuart points to the example of YouTube’s recommender algorithm, which reportedly nudged users towards extreme political views because that made it easier to keep them on the site. We already see AIs misbehaving for this reason. That’s true even if the goal isn’t what we really want, or the methods it’s choosing are ones we would never accept. AI today lacks common sense, and simply does whatever we’ve asked it to. Unfortunately there’s a big problem with this approach: it’s incredibly hard to say exactly what you want.

This is so obvious it almost doesn’t even seem like a design choice, but it is.

In his new book, Human Compatible, he outlines the ‘standard model’ of AI development, in which intelligence is measured as the ability to achieve some definite, completely-known objective that we’ve stated explicitly. Stuart Russell, Professor at UC Berkeley and co-author of the most popular AI textbook, thinks the way we approach machine learning today is fundamentally flawed. So it’s kind of like saying you’ve come across a terrible accident and you say, “Well, no one should call an ambulance because someone’s going to call an ambulance”. …if no one’s allowed to talk about the problems, then no one is going to fix them.

World in conflict unlimited reinforcement points