Peter Voss: The Third Wave of AI

All right. Okay. What I'd like to talk about is the third wave of AI. And what I mean by that is human-like intelligence. So what do I mean by intelligence or what do we mean by intelligence? What do we expect of an intelligent being?

We expected to be able to learn interactively in real time, you know, unsupervised without a teacher, just learning from the interaction, to be able to understand what's going on, to understand the context, to make sense of it, to be able to remember what was, what happened or what was said two seconds ago, two weeks ago, and to be able to use that in your interaction with the world, to be able to reason to apply reasoning to the situation and to be able to adapt and modify according to, you know, ongoing circumstances.

So my, my talk will be, my example will be centering on conversational AI because that's my, that has been the focus of my development, but really what this intelligence and third wave relates to is everything, you know, robotics, any, anything we are intelligence is, is employed, but specifically in conversational AI, there are so many different applications where you really want intelligent interaction in, in your, in your machine, in your, in, in your AI.

So whether it's for eCommerce, where you're interacting, you know, with customers and you want to give them support, you want to offer them something. You want to have an intelligent conversation as a front end to software. If you're interacting with ERP software system, you ideally just want to tell the system what you want.

You know, give me the sales for the last three months in Sweden, and then you wanna follow up and say, okay, how did that compare to last year? And without having to go down menus and selecting things, you, you know, the system should just give you that information. I don't have time to go through all of the, the use cases.

You know, whether it's VR or AR or an automotive or in robotics, if you have a robot in a hospital or a hotel, the hospital, you want be able to tell it, just go to the dispensary and pick up this order and deliver, you know, to this patient or doctor or in a hotel. You know, please bring me a shower cap tomorrow morning. I want two eggs over easy. So those all require intelligent interactions. Now the problem is we don't have that in our current chat bots.

You know, the chat bots provided by all the big companies and hundreds of small companies really have this, this big problem. They don't remember. They don't remember what you said two seconds ago. Nevermind what you said last week. They cannot learn interactively.

They are, they are taught at the factory and then it's a read only system. They don't have deep understanding. They just have pattern matching. So if you say I hate Uber don't ever gimme Uber again, it'll probably still trigger the Uber app and they can't reason they can't, you can't really have an ongoing conversation. So it's also very hard to personalize this to the individual to Ize it. I'd like to show you state of the art system of how it performs and then compare that to what's possible with third wave approaches.

So here's Alexa, which everybody knows trying to do something relatively straightforward. All right, Alexa, Alexa, add chips and guac napkins and three bottles of wine to John's birthday list. I added chips, guac napkins, three bottles of wine to your shopping list. Actually, Alexa, remove the napkins from the list To clear your list or remove an item. Visit the Alexa app.

Okay, then Alexa, what do I have on John's birthday list? You Have one item on your shopping list.

Chips, walk, napkins, three bottles of wine. You call all that one item. Alexa. Remember that John's favorite cake is red velvet. I'm not quite sure how to help you with that. Yeah.

So, you know, there's no understanding. Alexa doesn't know which list you're talking about, which items you're talking about. And this is in spite of thousands of people working on this, on this technology. And I'll explain what, why these limitations there, same contrast. I'd like to show you what we did is we hooked up our brain, a third wave brain to the Alexa microphone and speaker. All right, let's make this happen.

I go, Hi there. What's on your mind. Let's Add chips and guac napkins and three bottles of wine to John's birthday list. Good morning, Sam. I don't know what guac is.

Oh, guac is the same thing as guacamole. Okay. Creating John's birthday list. Is that okay? Yep. I added chips and guacamole napkins and three bottles of wine to John's birthday list. What was the food on that list? Again? Chips and guacamole and three bottles of wine. Actually let's make that five bottles. Sure. The list is now five bottles of wine chips and guacamole and napkins. Great. Let's send it over to June and ask her if there's anything I'm missing, Please sending John's birthday list to June via email.

Wait, always contact June by Skype. Okay. Sending the list to June via Skype. Remember that John's favorite cake is red Velvet. Got it. Fantastic. See this very different experience can, can learn if it doesn't know a term like guac, same as guacamole and use that and you know, really understand deeply what's what's going on. So what is the problem? The problem is that current chatbots don't have a brain. So what we really need is we need a cognitive engine that can think and reason and learn in real time. And this is where basically the third wave of AI comes in.

This term was, was used by DPA in several presentations. And what they mean by the three waves is the first wave is basically what people also call good old fashioned AI. So that's all the work in AI that was done over the decades. Mainly logic, logic based programming, expert systems, you know, flow, charting, some statistical techniques and so on.

And, and this is really what was driving, you know, for example, a deep blue chess champion. And then we, the second wave hit us about eight years ago, nine years ago, when companies figured out how they could use massive amounts of data, massive amounts of computing power to make neural networks really work, deep learning machine learning. And that's, it's been a real revolution.

It, it definitely has revolutionized speech recognition has become much, much better object recognition, you know, sentimental as and so on. And, and of course what's driving this as well, is the, the ability to more accurately target advertising.

You know, this is what makes us worth billions of dollars and why so much money. One of the reasons so much money went into deep learning machine learning in the early days, but it's also become quite apparent what the limitations are of deep learning machine learning of the second wave, you know, to the extent that godfather of, of deep learning Jeff end and a few years ago said my view is to throw it all away and start over. That's how frustrated he was limitations or the head of Google. Deep mind.

Deep learning is an amazing technology, but definitely not enough to solve AI, not by long shot. Now this is quite a remarkable statement by the CEO of a company that employs five, 600 PhD level researchers in deep learning machine learning. So clearly there are severe limitations in this approach. So what the third wave is it's inherently looking at what does intelligence require? It's a cognitive architecture that you have all the components required to think and reason and learn and, and do so interactively do that in real time.

So I started working on this about 20 years ago and I, you know, I came up with a design for an highly integrated cognitive architecture. Cognitive architectures have been around for quite a long time, but they haven't really worked that well.

And for, for several reasons, which we discovered you need a very high performance knowledge graph, and the components need to be very highly integrated. I don't have time to go into that, but with a properly designed cognitive architecture, you can get this deeper understanding and, and this interactive memory and contextual reasoning and, and, and so on.

Now, contrast that to the way chatbots are designed today using first and second wave technology. So pretty much all the chatbots out there that you come across, use a big categorizer. So whatever the input is, it, it puts us into a certain category. What you want to know, blah, blah, blah, umbrella. Okay. You want to know the weather and, you know, I hate Uber.

Oh, you probably want Uber. And then somebody designs basically some kind of a flowcharty program to go through the steps. Okay. Where do you want to go? How many people are going, do you want UberX?

And, and that's basically it, there are some very sophisticated tools that help design these, these systems now, but that's, it, there really is no intelligence. So to contrast these two approaches first and second wave on the one hand versus third wave is really like chalk and cheese. It's a fundamentally different approach. You have interactive learning versus just factory learning. You only need a small amount of training data because you have an ontology that has the hierarchy hierarchy of knowledge.

Again, I don't have to go time to go through all of, all of the details. One of the other, I think important differences is that with deep learning machine learning, you have a black box.

It's, it's not the system is not screwable you basically, if some, if it doesn't behave the way you expect it to your only remedy is really to throw more training data at it and, and hope that that will fix the problem without breaking something else without catastrophic forgetting. Whereas with the third wave within knowledge graph based system, ontology based systems, the system is fully screwable.

If it, it, it, you know, doesn't behave the way you expect it to, you can actually get to the bottom of and find out, is it some knowledge that's missing? Is it something in the reasoning that's missing or, or whatever it is, and you can specifically address the problem then.

So it, it, it really requires a fundamental rethinking of, of, of the, of the approach. And it it's a, it's a big problem. I I'll talk a little bit more about that, but I think you could see now how the cognitive, how cognitive assistant can really fundamentally revolutionize the way we interact with, with machines, with software that we can have this hyper personalized assistant that learns about our particular situation, our preferences, our history, and then can, can give us, you know, much more useful interaction.

So I'll, I'll finish off by just addressing the question of, you know, why, why don't we have the future yet? You know, we expect to have, if we think about movies of decades ago, you know, we have AI that you can talk to and understands you. So why don't we have that yet? So the another way of looking at that is up to now, AI really has been narrow AI. So when the, when the coin artificial intelligence was coined 60 some 60 odd years ago, the idea was to build the thinking machine, a machine that can think and learn and reason the way humans do now that turned out to be really, really hard.

So what happened is AI turned into narrow AI, and that's really what we've been doing for the last 50 years. And that's what first and second wave do. Your intelligence is really external. It's the program as intelligence or the, the data scientist intelligence that figures out how we need to arrange the, the, the, the data, what algorithms we need to write. So for example, again, deep blue and, and chess playing, what do we need to do to write the algorithm that can play chess really well? Not how do we build a thinking machine and the same way as a go champion?

You know, how do we optimize the system that can play go? And it's really, we need to get away from that. And the third wave is the way of thinking about that, of having general intelligence, the intelligence resides in the machine itself and can learn and become smarter as it goes along without, you know, a program having to figure out or data scientist having to figure that out. So that's really what the third wave of, of AI is about, or another way of describing this artificial general intelligence. So artificial general intelligence, cognitive architectures really go together.

Now, the reason we, we, we don't have more progress in the, in the third wave of cognitive architectures is what I call the narrow AI trap. And that is under commercial pressures, you know, next quarter results, or getting a product out out into the field quickly. It is almost always easier to take the program as intelligence or a data scientist intelligence, and figure out how to solve a particular problem. So people who've tried to build AGI or to build general intelligence, cognitive architectures come under tremendous pressure to basically just take shortcuts and hard code things.

And, and basically then they never get, never get back to solving the big problem. So you really have to have that, that, that vision of what needs to be done to get systems, to be truly intelligent. And even though my examples have all been in conversational AI, the same thing really applies to, you know, vision and robotics.

You want a robot to be able to learn interactively as it interacts with the world, to have understanding of the situation, to use context, to have memory in, in vision, a child can see one single photograph of, of an elephant and has never seen an elephant before Jaff and be able to recognize it without supervised training. You don't even have to tell them that it's you off. They know it's a different kind of animal. So that's the kind of intelligence that you want, where you have unsupervised learning. So I'll leave it at that.

And you can read up a lot more about, about the third wave and artificial general intelligence and, and our project at our company's website. I've written quite a lot of articles about AI and AGI and, and also ethics of, of AI. So I'm open to taking questions right now. Thank you.