The Illusion of Choice: Protecting Against AI-Driven Influence and Manipulation [Introductory]

Amazing. So, hi everyone. I'm Sam. I work for Vodafone Group as a product manager. But today I'm not going to be talking about Vodafone or really my day job, right? I'm going to be talking about something which I think is very pertinent and important to every single person in this room. And that is this concept of free will.

So, free will is the ability to act and make choices independent of any outside influence. And I'd like everyone in this room to ask themselves a question in their head, okay? Does everyone in this room consider themselves to have free will? And the reason I ask this question is because I think that today our choices are already being influenced by third parties and organizations.

So, we live in a hyper-digital era, okay? So, everything that I do and the choices that I make are influenced by what I see on the internet and through my mobile phone.

So, the places that I go, the restaurants that I eat at, the hotels that I book are influenced by the recommendations that Google, that Booking.com gives me, okay? The opinions that I form and the news that I read are influenced by Google, by TikTok, by Amazon, by all these different newspapers that we get our information from. And the things that I buy, all of these are influenced by recommendation engines that think they know what I might want to buy next.

And even the TV shows that I watch, or Joshua watches, I'm not sure why it says Joshua on there, but even the TV shows that I watch, okay, are influenced by these recommendation engines. So, today, all the choices that we make are already being influenced by organizations and third parties. But I'd argue that sometimes these kind of recommendations and this level of personalization actually isn't that good.

But still, our digital identity and those different versions of us are readily available on the internet and they are being used to personalize our experiences. And just some stats for you. There is a data broker in the U.S. that has data on 2.5 billion people. 2.5 billion. And on each individual person, they have up to 11,000 attributes on every single one of them that can be inferred from the data that they have.

And these can range from anything like your age, your address, to what type of dog you like, to your favorite color, to your religion, to your political persuasion, even to your sexual orientation. All of this is available in the attributes that these data brokers have.

So, our data is available for loads and loads of different people to access. We've been leaking our data for years and this data is being used to personalize our experiences today. And some of that personalization, to my earlier point, I don't think it's that good today.

So, some of these Amazon recommendations that I get are really obscure and I don't actually buy a lot of the recommendations they give me. But the argument I want to make and what I want to discuss today is that, actually, when we take generative AI and we take this ability to hyper personalize experiences, I think we end up with something like this. I think that our customer experiences, every website we go to, everything we access will be hyper personalized to each individual one of us. And I think it's going to bring a lot of benefits.

So, our healthcare will be able to go through an automated system that's actually useful, right? We're not going to have to spend ages talking to some person on the phone. They will know exactly what our healthcare needs are and they'll be able to triage us to a doctor super easily. Our fashion, right? If we're going to a conference and perhaps, like me, you need a new blazer, you might see yourself wearing that blazer on a relevant fashion or shopping website.

So, actually, things are going to get much better. And the ads that we get are going to be designed to appeal to our specific emotional triggers.

So, leveraging those 11,000 attributes and leveraging generative AI, our customer experiences are on the surface going to get a lot more tailored and a lot more personal to each individual person in this room and around the world. But I would encourage us to ask a question. Will this level of personalization be done for our benefit? Or will it be done to influence us towards an outcome that doesn't actually benefit us, but it benefits the organization that's building that customer experience?

So, in this presentation today, I'm going to speak about the design techniques that organizations are using today to influence us. I'm going to speak about the implications of gen AI on those design techniques. And we're going to talk about regulation and how identity fits into all of this.

So, one of the techniques that I think is most prevalent is something called a dark pattern. So, every mobile game that you go on, 97% of mobile games, have at least one dark pattern. And this is effectively when the designer of a game makes an active design decision to influence you into making a decision. That might not be to your benefit.

So, in a mobile game, for example, if you don't pay for some gems, or if you don't pay to unlock something, it could take you a really long time to rebuild your base or to beat your friends. So, if you don't pay for these gems, actually, you might lose the game. And this is really big business, right?

So, 97% of games have at least one dark pattern. And the mobile game market is worth 92 billion dollars. And most of that comes from microtransactions.

So, these transactions and these design decisions that these kind of games are making are working today. And this example is very personal to me, actually, because in my younger years, I won't tell you what age this was at, I actually spent 300 pounds of my dad's credit card money on the game that you see on this screen.

So, I've been influenced by this game in my younger years. Maybe I was a bit more naive than I am today into making a decision, okay? And it's not just dark patterns. There are other tried and tested design decisions that websites make to push us towards an outcome.

So, choice architecture. When you go to booking.com, you might see that the top three ads are really bold. They really jump out of the screen at you, okay?

So, there's some tags on there, they're in different colors, and they really jump out of the screen. And if you want to filter, there are literally hundreds and hundreds of different filters and different options.

So, the path of least resistance for us as consumers is to click on one of those top three options. And these design decisions are working.

So, on Google, on Amazon, on booking.com, a third of all clicks goes towards the first one or two rows of products. So, choice architecture isn't about telling you what to do, but actually what they do is they organize the context in which people make decisions, so you're super likely to pick what they want you to pick. And for the eagle-eyed amongst you, you might have noticed the dark pattern.

So, they organize your context in a way, and they make those hotels super bold. And what they also do is they also say there are only three rooms left at this price on our site.

So, they're organizing the context in which you make decisions, and they're pressuring you to make that decision super, super quickly. Now, what we see here is done at the generic level, okay?

So, there's not that much personalization being done with our 11,000 attributes. But I think that when we take generative AI, and when we think about this context of hyper-personalization, we get to something that actually looks a bit like this.

So, those 11,000 attributes of my attributes in this example are taken and fed into kind of a gen-AI engine, and the exact same content, so the exact same hotels, are presented in such a way that they look more appealing to me based on my specific attributes and emotional cues. So, they're exactly the same hotels that are presented in a way that's more specific to me.

So, maybe they know that I really value a city view. Maybe they know I really like balconies, okay? Maybe they also know I'm more likely to click on a picture with a sunset, as you can see. Maybe they also know that I really value reviews from people who are the same age as me. And what they do is they use gen-AI to cheaply reorganize the positioning of the pictures. Maybe they also generate a little bit of content, say, Sam, this is voted the best area in Berlin by people your age. Maybe they also say, Sam, it's near your favorite bar.

And to finish it off, they'll also say, instead of just six rooms being available on this site or being booked today, they'll say three colleagues attending EIC booked this hotel today. So, they take the same options and they present it in a way that's more appealing to each individual one of us.

So, maybe instead of having a third of clicks going towards the first one or two sites, we'll go to 60%, right? So, we will click on the ads that these websites want us to click on because it's more appealing to each individual one of us. And you might be thinking, Sam, surely this is a little bit questionable and probably there's regulation in place. And what if I told you that today, even with GDPR, your data and the data of everyone in the EU is being exposed on average 376 times per day to third parties. Today.

So, there's a system called real-time bidding, which is effectively an online auction. Every time you go to a specific website that has programmatic ads, that website will take your data and it will instantly send it to lots of different third parties. And when I say data, I mean stuff around who you are, what you're reading, what you're watching, your interests, your habits, even your religion. These websites might have fruit cookies and they send that data to lots of third parties. And those third parties then bid on the chance to get on that webpage that you're on.

So, this is a real gray area. And today, your data is being leaked 376 times per day. And this is big business. We're talking billions and billions of transactions happening in this space every single day.

So, even today, there are systems that get around this kind of regulation that we have in place. And GDPR is well known, right?

So, even today, our data is being leaked. And there has been some effort made to try to mitigate the risks that generative AI kind of approaches, right? And the EU AI Act, I had to practice that loads of times before this talk because it's a real tongue twister. The EU AI Act bans using AI for manipulation that can cause significant harm. But I would pose a question once again. Do any of those use cases that we spoke about actually cause significant harm? Or are we just buying another pair of shoes, right?

And they also ban AI for subliminal, using AI for subliminal techniques that influence behavior. But none of those techniques that we've spoken about are subliminal. They can be noticed if you know what to look for. And to give an example, dark patterns, they're ethically questionable. And they've been banned to an extent in games, but they're still openly used on websites.

So, with gen AI, the level of personalization, the level of influence is going to increase dramatically. And we're going to enter even more of a gray area.

So, the question I think we have to ask ourselves is, where is the line between persuasion and manipulation? How many attributes can be used before we step over into the latter?

So, this is a real challenge. We're entering this new kind of age where things are going to get a lot more kind of covert, right, with these new technologies. And regulation currently does not go far enough. There are other ethical guidelines that have been kind of spoken about across the EU and globally, but no one's really cracked it, right?

So, no one has really addressed any of those use cases that we've spoken about today. It might sound like I'm kind of giving you guys like an end-of-the-world kind of speech, right? And that is not my intention, to be clear. Because I think that everyone in here could have a hand in shaping the future and making sure that we're able to get benefits from these new systems.

So, I think of the solution to this kind of problem we've spoken about in terms of three key areas. Firstly, control.

We, as users and as consumers, need to have control over when, where, and how our data is being used. So, how those 11,000 attributes are being leveraged. And we need to be able to dictate who can access it and be able to revoke their access super easily. We also need control over when our experience is being hyper-personalized and to be able to toggle easily between that level of personalization. And we don't need some kind of cookies type model, right? We need full, easy control over the personalization that's going on and the data that's being leveraged.

And everything I've spoken about for control, we also need transparency on. So, we need transparency into when, where, and how our data is being used, how our experience is being personalized, and when AI is being leveraged.

So, there's definitely been some talk around labeling AI images. We absolutely need that to happen in order for us to survive in this new world. And I think, and this is quite an important topic for all of us here, that to solve those two areas, decentralized identity could be a solution. If we can take away ownership of data from centralized authorities and give it back to the hands of our users, and if we can have a full blockchain ledger, for example, where we can see the usage of all those attributes, that could be really, really powerful.

And I was just in a talk around kind of the EU IDAS initiative that's coming into place across Europe. And one of the things they were speaking about was electronic attributes, right?

So, it's not just about kind of verifiable credentials. It goes beyond that to actually ensure that we can also have control over our attributes. It doesn't go far enough yet, but maybe in the future we need to expand out the scope of those electronic attributes, right? To include some of those behaviors and some of those attributes we've spoken about before. And everything I've spoken about, right, has to be underpinned by clear regulation. We need to draw a clear line between persuasion and manipulation.

We need to not just advise companies to be ethical, we need to actually force them to be ethical with these new technologies, right? And we need to also enforce the use of decentralized identity and other controls in this new kind of world that we're entering, right?

So, these three key things, I think, could help us address the risks associated with GNI hyper-personalization and this concept of influence. Now, I've waffled on for ages, right? But what does this actually mean, okay? In summary, I think that there is a massive opportunity for organizations to build hyper-personalized, hyper-tailored customer experiences that are actually genuinely great, right?

So, Sam can get some fashion advice because he's got terrible fashion sense. We can easily go to a doctor's, right? And we can kind of access content that's really personalized to us.

So, there's a lot of benefits. And for organizations, they should be able to really leverage these technologies as a force for good. But as we've discussed, there are techniques being used as a force for evil today.

So, choice architecture and dark patterns, these have been leveraged today to influence us to the benefit of an organization. And if we don't put the proper regulation in place, if we don't put the proper controls in place, and if we don't enable transparency into everything that's going on, I would encourage us to ask, to re-ask that first question, right?

Today, personalization is quite obvious. In the future, it might be much more effective and less obvious.

So, if we continue in this direction, will we still have free will? Thank you.