You have a very precisely defined goal and you build a machine that's superhuman in its capabilities for achieving goals. If it turns out that the subsequent behavior of the robot in achieving that goal was not what you want, you have a real problem.

A lot of people talk about sometime around 2030, machines will be more powerful than the human brain, in terms of the raw number of computations they can do per second. But that seems completely irrelevant. We don't know how the brain is organized, how it does what it does.

There is no scientific theory that could lead us from a detailed map of every single neuron in someone's brain to a conscious experience. We don't even have the beginnings of a theory whose conclusion would be such a system is conscious.

Google or other search engines are examples of AI, and relatively simple AI, but they're still AI. That plus an awful lot of hardware to make it work fast enough.

I expect that people are going to feel differently about that once they're aware that AI systems can watch through a camera and can, in some sense, understand what it's seeing.

Its really important to understand the difference between sentience and consciousness, which are important for human beings.

To my knowledge nobody - no one who is publishing papers in the main field of AI - is even working on consciousness. I think there are some neuroscientists who are trying to understand it, but I'm not aware that they've made any progress.

When people talk about the singularity, when people talk about superintelligent AI, they're not talking about sentience or consciousness. They're talking about superhuman ability to make high-quality decisions.

The singularity has nothing to do with consciousness.

AI's ability to recognize visual categories and images is now pretty close to what human beings can manage, and probably better than a lot of people's, actually. AI can have more knowledge of detailed categories, like animals and so on.

I used to say that if you gave me a trillion dollars to build a sentient or conscious machine I would give it back. I could not honestly say I knew how it works.

Most of the AI goes into figuring which are the important pages you want. And to some extent what your query means, and what you're likely to be after based on your previous behavior and other information it collects about you.

No one has a clue how to build a conscious machine, at all. We have less clue about how to do that than we have about build a faster-than-light spaceship.

The robot is not going to want to be switched off because you've given it a goal to achieve and being switched off is a way of failing - so it will do its best not to be switched off. That's a story that isn't made clear in most movies but it I think is a real issue.