Yuval N Harari on the Promise & Peril of AI

Does the rise of AI mean decline – even the end – of Homo Sapiens? Yuval N Harari discussed
the outlook with WSJ at the Wall Street Journal’s CEO Council Summit in London.

1. Bringing up Baby

WSJ : You call AI (or alien intelligence) the rise of a new species that could replace Homo Sapiens.

Harari: For the first time, we have a real competition on the planet.

And now we are creating something that could compete with us in the very near future.

Most important thing to know about AI is that it is not a tool, but it is an agent.

It can make decisions independently of us. It can invent new ideas. It can learn and change by itself.

All previous human inventions, whether the printing press or atom bomb, are tools that empower us.

WSJ : They needed us.

Harari: They need us because a printing press cannot write books by itself and it cannot decide which books to print next.

An atom bomb cannot invent the next more powerful bomb and cannot decide what to attack.

An AI weapon can decide by itself which target to attack and design the next generation of weapons by itself.

WSJ:Your argument is that we, especially the powerful leaders in this room, have a lot of responsibility, because how we act is how AI will be, like a baby.

Harari: There is a big discussion around the world about AI alignment: We are creating these increasingly superintelligent, very powerful new agents. How do we make sure that these agents remain aligned with human goals and with the benefit of humanity, that they do what is good for us?

The two main problems: First, the very definition of AI is that it can learn and change by itself. So when you design an AI, by definition, this thing is going to do all kinds of things which you cannot anticipate.

The other, even bigger, problem is that we can think about AI like a baby or a child. And you can educate a child to the best of your ability. He or she will still surprise you for better or worse. No matter how much you invest in their education, they are independent agents. They might eventually do something which will surprise you and even horrify you.

The other thing is, if you tell your kids, “Don’t lie,” and your kids watch you lying to other people, they will copy your behavior, not your instructions. Now if we have now this big project to educate the AIs not to lie, but the AIs are given access to the world and they watch how humans behave and they see some of the most powerful humans on the planet, including their parents, lying, the AI will copy the behavior .

2. Everything, everywhere

WSJ: Can you speak to them as if we were sitting here 36 months from now? Is there any world in which AI doesn’t have a significant impact on their business?

Harari: So we now know that the Industrial Revolution and trains, they completely transformed everything. But it just took more than five years. The same is likely to happen with AI in all fields, from the obvious to the less obvious.

I think that one of the first fields we’ll see major changes in is finance, that AI is going very quickly to take over the financial system. Because finance is purely an informational realm. You don’t see these tens of thousands of self-driving vehicles yet. The problem is that for driving, you need to deal with the messy, physical world of pedestrians and holes in the road and whatever. But in finance, it’s only information in, information out. It’s much easier for an AI to master that.

And what happens to finance once AIs, for instance, start inventing new financial devices that the human brain is simply incapable of dealing with because it’s mathematically too complex?

3. A Useless Class

WSJ: Let me get back to what you’ve said about replacing jobs. You’re worried about what becomes a useless class. What do we do to make sure we, as a society, not only survive, but thrive?

Harari: You can use the same technology to create completely different kinds of societies. We saw it in the 20th century— we used exactly the same technology to build communist  totalitarian regimes and liberal democracies.

We have a lot of choices about what to do with it—if we remember that for the first time, we are dealing with agents and not tools, so it makes it much more complicated. But still, most of the agency is in our hands. And the question of how we develop the technology and, even more importantly, how we deploy it, we can make a lot of choices there.

The main problem is that now the companies and countries that lead the AI revolution have been locked into an arms-race situation. So even if they know that it would be better to slow down, to invest more in safety, to be careful about this or that potential development, they are constantly afraid that if we slow down and they don’t slow down, they will take over the world.

Homework : What is MCP (Model Context Protocol) ?