WEBVTT 00:00:13.900 --> 00:00:16.700 Welcome to the KuppingerCole Analyst Chat. 00:00:17.200 --> 00:00:17.500 I'm your host. 00:00:17.700 --> 00:00:19.100 My name is Matthias Reinwarth. 00:00:19.200 --> 00:00:22.100 I'm Advisor and Analyst with KuppingerCole Analysts. 00:00:22.100 --> 00:00:25.000 And so is Jonathan Care. 00:00:25.200 --> 00:00:28.600 He is hailing from Portugal, close to Porto. 00:00:29.200 --> 00:00:29.900 Good to have you. 00:00:30.400 --> 00:00:30.800 Hi, Jonathan. 00:00:32.200 --> 00:00:32.700 Hi, Matthias. 00:00:33.100 --> 00:00:36.100 And thank you for having me on the chat again. 00:00:37.000 --> 00:00:37.500 Right. 00:00:37.600 --> 00:00:39.200 So how is weather in Portugal? 00:00:39.500 --> 00:00:40.600 How are you doing in Portugal? 00:00:42.000 --> 00:00:42.900 Doing pretty well, actually. 00:00:44.700 --> 00:00:48.613 We've just started going into the warmer 00:00:48.613 --> 00:00:53.700 season, so that basically means it's t-shirt weather 00:00:53.700 --> 00:00:54.500 once again. 00:00:55.500 --> 00:01:01.700 And I'm hoping my roofer finishes the roof repairs so I can get my solar panels back 00:01:01.700 --> 00:01:04.400 and stop paying for electricity. 00:01:04.800 --> 00:01:12.600 And so, yeah, the basics also are valid in Portugal, unfortunately. 00:01:12.800 --> 00:01:17.000 So you need to take care of electricity and weather and the roof. 00:01:17.900 --> 00:01:25.500 Yeah, no, it's very agreeable here and looking forward to the season getting warmer. 00:01:26.300 --> 00:01:29.400 And do I believe you or don't I believe you? 00:01:29.400 --> 00:01:31.900 That's the question that we actually want to talk about today. 00:01:32.000 --> 00:01:36.600 We want to talk about misinformation, disinformation, untrust. 00:01:37.400 --> 00:01:43.500 And this is something that will be a key topic also at our event in May for EIC. 00:01:43.800 --> 00:01:45.400 But it's a key topic for everybody. 00:01:45.400 --> 00:01:54.600 So to have a proof that it's relevant, can you give, just for the recent months or years, 00:01:55.000 --> 00:02:01.400 some examples where misinformation and disinformation really spreads and really does harm? 00:02:02.600 --> 00:02:06.114 I don't want to delve into the complex, 00:02:06.114 --> 00:02:10.800 intense geopolitical situation we all find ourselves 00:02:10.900 --> 00:02:13.900 in, but let me give you one example. 00:02:14.900 --> 00:02:23.200 I had a conversation with the General Medical Council, who are the organization in the UK 00:02:23.200 --> 00:02:30.900 that regulate and can disbar doctors from practicing. 00:02:31.200 --> 00:02:37.700 So they are the legislative board and have, yeah, they have legal powers. 00:02:40.100 --> 00:02:45.203 They said to me, this is during COVID, that 00:02:45.203 --> 00:02:50.900 they saw doctors going on Twitter and on YouTube 00:02:52.900 --> 00:02:57.700 and saying that, you know, vaccines were not effective. 00:02:58.500 --> 00:03:00.550 And, you know, I think one of them was saying 00:03:00.550 --> 00:03:02.600 we should all just drink chocolate milkshakes 00:03:02.600 --> 00:03:03.900 and that would make it better. 00:03:03.900 --> 00:03:10.100 Without delving into the rights and wrongs of vaccines, and I know there are strongly 00:03:10.100 --> 00:03:20.400 held beliefs on that, the General Medical Council's view was that it is a matter of 00:03:20.400 --> 00:03:25.900 medical policy that vaccination is effective and that doctors should be following. 00:03:26.800 --> 00:03:29.800 And failure to do so is a fitness to practice issue. 00:03:29.800 --> 00:03:31.400 In other words, they can lose their license. 00:03:32.900 --> 00:03:39.000 So the reason they contacted me is they said the defense they were getting in a number 00:03:39.000 --> 00:03:41.800 of cases was that, oh, God said that wasn't me. 00:03:41.900 --> 00:03:42.900 It must have been a deepfake. 00:03:43.800 --> 00:03:51.500 And so they were looking for any evidence, anything I could provide in terms of tooling 00:03:52.200 --> 00:03:57.100 or experiential knowledge to identify a deepfake. 00:03:57.100 --> 00:04:04.200 And as we know, the quality of deepfake videos has only got better. 00:04:04.700 --> 00:04:09.293 So last year at EIC, I presented a beautiful 00:04:09.293 --> 00:04:14.200 photo of me and the Dalai Lama sharing a roller 00:04:14.200 --> 00:04:15.100 coaster ride together. 00:04:15.900 --> 00:04:20.300 And obviously, anyone who knows me knows I don't like going on roller coasters anymore. 00:04:20.300 --> 00:04:21.400 So that was obviously a fake. 00:04:22.400 --> 00:04:25.800 Really, the Dalai Lama and I were actually eating spaghetti bolognese together. 00:04:26.800 --> 00:04:30.711 So in all seriousness, the use of tools such 00:04:30.711 --> 00:04:34.800 as NanoBanana and even the recent developments 00:04:35.300 --> 00:04:45.300 with Anthropic Claude and the innovations with OpenAI and Sora 2 have only improved 00:04:45.300 --> 00:04:48.700 video generation to make it even more lifelike. 00:04:48.700 --> 00:04:53.000 And so it is going to, you know, these deepfakes are a challenge. 00:04:54.200 --> 00:04:58.506 If you like the ticks, we would say, you know, 00:04:58.506 --> 00:05:03.000 like the uncanny valley, which is the perception 00:05:03.800 --> 00:05:08.467 of a person that doesn't quite look human, 00:05:08.467 --> 00:05:13.800 raises some, yes, odd ancestral fears within us. 00:05:13.900 --> 00:05:17.500 And so we can all spot something that isn't quite human. 00:05:17.500 --> 00:05:21.500 The hands look wrong or there were too many fingers and so on. 00:05:21.500 --> 00:05:25.300 All of these errors, all of these ticks are disappearing. 00:05:25.600 --> 00:05:29.085 And so we are seeing AI-generated imagery, 00:05:29.085 --> 00:05:33.400 including motion imagery, that is better and better. 00:05:33.900 --> 00:05:40.500 And I think that if we look at this information, we frame it as a political problem. 00:05:40.600 --> 00:05:44.500 So we think about elections, propaganda, and social media. 00:05:45.600 --> 00:05:48.400 Well, as I said, that's already out of date. 00:05:49.900 --> 00:05:53.050 AI has industrialized production of false content 00:05:53.050 --> 00:05:56.200 and moved the attack surface into the enterprise. 00:05:56.700 --> 00:06:00.063 So, for example, you know, I would, you know, 00:06:00.063 --> 00:06:03.500 if I touch up a video to make me look handsome 00:06:04.200 --> 00:06:10.800 and windswept and interesting, you know, is that false content or is it just me making 00:06:10.800 --> 00:06:12.500 the best of my appearance? 00:06:12.500 --> 00:06:18.900 What was once a societal concern is now a direct financial security risk. 00:06:19.400 --> 00:06:22.760 If you think about the number of processes 00:06:22.760 --> 00:06:26.600 that rely on us going, oh, yes, that's Matthias. 00:06:27.200 --> 00:06:31.200 He's allowed to, for example, access the Treasury accounts. 00:06:31.900 --> 00:06:33.500 That's Jonathan Kaye. 00:06:33.500 --> 00:06:37.110 He's allowed to access the safe where the 00:06:37.110 --> 00:06:41.600 company's private keys and certificates are stored. 00:06:42.800 --> 00:06:45.600 So the numbers make a point. 00:06:46.300 --> 00:06:53.600 Deepfake fraud losses in North America alone exceeded $200 million in quarter one of 25. 00:06:53.900 --> 00:06:55.400 That's one quarter. 00:06:56.000 --> 00:07:01.300 And it is my belief that's only increased since then. 00:07:01.300 --> 00:07:02.300 Yeah, I think so. 00:07:02.700 --> 00:07:09.400 And I didn't think of deepfakes being the modern version of the dog ate my homework. 00:07:09.700 --> 00:07:13.100 So this is interesting to hear. 00:07:14.200 --> 00:07:17.400 But, yeah, of course, this is something where it wasn't me. 00:07:17.500 --> 00:07:19.600 It was somebody else who fabricated something. 00:07:20.100 --> 00:07:27.100 That is even another dimension of this disease that's spreading. 00:07:27.600 --> 00:07:29.832 When I did the intro a few minutes ago, I 00:07:29.832 --> 00:07:32.500 said misinformation and disinformation and trust. 00:07:33.000 --> 00:07:38.400 And this is something that comes out of my mouth as if it was one term, but it's three. 00:07:39.600 --> 00:07:43.503 Maybe to open the discussion, what is even the 00:07:43.503 --> 00:07:48.000 difference between misinformation and disinformation? 00:07:48.200 --> 00:07:50.100 And what should we know about the two of them? 00:07:51.000 --> 00:07:52.400 I think that's a great point. 00:07:53.400 --> 00:07:59.700 For example, misinformation is false information shared without malicious intent. 00:08:00.300 --> 00:08:07.002 And I use the example of my uncle who will forward me a 00:08:07.002 --> 00:08:14.800 health claim or will forward me something he genuinely believes. 00:08:15.400 --> 00:08:19.600 This information, on the other hand, is deliberately crafted to deceive. 00:08:19.600 --> 00:08:23.478 The tricky part is that disinformation often becomes 00:08:23.478 --> 00:08:28.400 misinformation once real people pick it up and share it sincerely. 00:08:29.300 --> 00:08:33.038 For example, someone could share with me that, I 00:08:33.038 --> 00:08:37.400 don't know, I'm trying to think of a ridiculous example. 00:08:38.700 --> 00:08:39.700 Most have already happened. 00:08:40.900 --> 00:08:44.300 Drinking chocolate milkshakes is a cure for COVID. 00:08:45.400 --> 00:08:51.400 And I might sincerely believe that I share that with you in good faith. 00:08:52.400 --> 00:08:55.300 Therefore, my act of sharing with you is misinformation. 00:08:56.600 --> 00:09:04.700 However, disinformation are the people that created that false information and shared it. 00:09:04.700 --> 00:09:08.700 And it may be to further political agenda. 00:09:09.000 --> 00:09:15.300 It may be, in that case, to increase the sales of their patent chocolate milkshakes. 00:09:16.000 --> 00:09:16.800 Who knows? 00:09:17.300 --> 00:09:25.100 But I think of this as like a counterfeit 10 euro note. 00:09:26.400 --> 00:09:28.100 A forger knows it's fake. 00:09:29.200 --> 00:09:34.700 Once it's in circulation, you, me and most people passing it along have no idea. 00:09:34.900 --> 00:09:39.203 So if we think about this in an organizational 00:09:39.203 --> 00:09:43.600 constant, this distinction drives our response. 00:09:44.600 --> 00:09:47.763 So inside a company, misinformation can be, for 00:09:47.763 --> 00:09:51.800 example, an employee sharing inaccurate threat intelligence. 00:09:52.000 --> 00:09:53.700 An area I'm very interested in. 00:09:53.700 --> 00:10:00.100 It can be a large language module, hallucinating facts in a board report. 00:10:01.300 --> 00:10:04.800 These things are AI governance and they are accuracy problems. 00:10:06.300 --> 00:10:11.200 Disinformation targeting the organization is an adversarial security problem. 00:10:12.000 --> 00:10:16.400 Threat actors, financial fraud, executive impersonation. 00:10:17.500 --> 00:10:20.900 Each of those has a completely different threat model. 00:10:21.400 --> 00:10:24.900 And most security teams are only thinking about one of it. 00:10:25.100 --> 00:10:27.700 And I think as you described, what is it? 00:10:27.700 --> 00:10:32.000 The question is how it is conveyed to those who are consuming this information. 00:10:32.200 --> 00:10:37.757 I think an important part is also the way it is presented in 00:10:37.757 --> 00:10:43.500 a way that it actually evokes emotions, often strong feelings. 00:10:43.500 --> 00:10:46.922 So that goes hand in hand to say this is not only 00:10:46.922 --> 00:10:51.600 something that is surprising or a fact that needs to be considered. 00:10:52.900 --> 00:10:57.600 Always some kind of, to put it mildly, engaging 00:10:57.600 --> 00:11:02.700 or it's really making people angry or very engaged. 00:11:03.000 --> 00:11:09.600 So it's not only the disinformation, the misinformation, but also the human factor, right? 00:11:10.600 --> 00:11:15.206 And again, here I'm going to continue to try and walk the 00:11:15.206 --> 00:11:20.700 political tightrope and avoid anything that's politically sensitive. 00:11:21.200 --> 00:11:22.400 Because there you go. 00:11:23.400 --> 00:11:25.700 Emotional resonance is beating factual accuracy. 00:11:25.700 --> 00:11:33.817 So we used to think of politics as rational people conducting 00:11:33.817 --> 00:11:41.800 reasoned debates and coming up with a policy, a way forward. 00:11:42.800 --> 00:11:49.052 Actually, what we're finding is that reasoned debate is actually 00:11:49.052 --> 00:11:55.500 less powerful, a convincer and motivator than emotional resonance. 00:11:55.900 --> 00:12:00.171 So a story that makes you angry or afraid 00:12:00.171 --> 00:12:05.900 travels faster and sticks harder than a dry fact check. 00:12:06.700 --> 00:12:10.900 So yeah, I like my chocolate milkshake thing. 00:12:10.900 --> 00:12:12.800 It doesn't make anybody angry or afraid. 00:12:14.000 --> 00:12:22.400 But if I preface it with the COVID vaccine is actually really bad for you. 00:12:22.600 --> 00:12:24.100 And many people believe this. 00:12:24.200 --> 00:12:31.900 I have a school friend who is quite convinced that COVID vaccine can 00:12:31.900 --> 00:12:39.600 give rise to heart conditions, can give rise to neurological issues. 00:12:39.600 --> 00:12:43.374 And so these are clearly things that you or 00:12:43.374 --> 00:12:47.500 I, well, certainly me, make me angry or afraid. 00:12:47.900 --> 00:12:50.000 Oh, my God, what's this vaccine? 00:12:51.400 --> 00:12:56.710 And even though I'm the rational part of me says, well, why 00:12:56.710 --> 00:13:02.200 would we have a vaccine that was designed to make people ill? 00:13:04.000 --> 00:13:10.100 Why would we have a vaccine that's designed to make people obedient to command? 00:13:10.400 --> 00:13:12.800 But these are things that make us angry, afraid. 00:13:13.500 --> 00:13:15.800 And I'm sure you've heard those rumors. 00:13:15.900 --> 00:13:18.700 I think when I mentioned that, I think I saw you nodding along. 00:13:19.000 --> 00:13:23.995 So those stories are familiar to us and they travel fast 00:13:23.995 --> 00:13:28.900 and they are much more than a dry fact check and so on. 00:13:28.900 --> 00:13:34.600 So it's very hard once these highly emotive stories are out. 00:13:35.500 --> 00:13:39.100 Debunking them is like trying to put toothpaste back in the tube. 00:13:40.100 --> 00:13:45.700 And one of my favorite authors, the late Terry Pratchett, had a phrase. 00:13:46.200 --> 00:13:52.400 He said, a lie can travel around the world while the truth is still getting its boots on. 00:13:52.800 --> 00:13:54.800 And it's this kind of idea. 00:13:55.800 --> 00:14:00.800 So more interestingly, what if we talk about pre-bunking rather than debunking? 00:14:01.400 --> 00:14:07.700 How can we inoculate people against these manipulation tactics before they encounter them? 00:14:08.200 --> 00:14:13.000 How can we teach them the pattern of the trick, not just correcting a specific lie? 00:14:13.300 --> 00:14:19.400 And again, this applies within organizations to security awareness training. 00:14:19.400 --> 00:14:22.739 Both of us have just been through, I think, 00:14:22.739 --> 00:14:26.700 some security awareness training from our employer. 00:14:27.900 --> 00:14:31.100 We teach people how deepfake fraud works. 00:14:32.100 --> 00:14:36.498 If you see a video from our CEO saying, hey, I need you to 00:14:36.498 --> 00:14:41.200 send me your password or, hey, I'm stuck in a foreign country. 00:14:41.200 --> 00:14:43.600 I need you to send me 500 euros. 00:14:45.000 --> 00:14:49.143 Teaching people how these deepfake fraud works before they 00:14:49.143 --> 00:14:54.000 encounter it is actually more durable than a policy that says, well, 00:14:54.000 --> 00:14:57.529 if anybody asks you to transfer money or give away 00:14:57.529 --> 00:15:01.200 your password, then you need to verify it and so on. 00:15:01.300 --> 00:15:04.245 Now, actually, the fact, again, that I'm sure many of us 00:15:04.245 --> 00:15:07.400 are listening to the podcast going, give away your password. 00:15:07.500 --> 00:15:08.700 I'd never do that. 00:15:09.700 --> 00:15:12.828 Good. That's an example of pre-bunking, 00:15:12.828 --> 00:15:17.800 inoculating, training people against that manipulative tactic. 00:15:19.200 --> 00:15:21.908 Before, and people like yourself and I guess 00:15:21.908 --> 00:15:24.800 myself as well, who are security practitioners, 00:15:25.000 --> 00:15:30.700 before we started engaging in security awareness training, there were a lot of people 00:15:30.700 --> 00:15:34.900 and there were studies done in Liverpool Street Station in the UK. 00:15:36.100 --> 00:15:38.820 People were offered a candy bar if they could 00:15:38.820 --> 00:15:41.600 supply their password, which they happily did. 00:15:41.900 --> 00:15:45.294 Now, fortunately, we have a security awareness 00:15:45.294 --> 00:15:49.500 mindset saying, no, actually, don't just believe a video. 00:15:50.000 --> 00:15:56.200 Don't believe an email saying, I need you to send me 500 euros. 00:15:56.800 --> 00:15:59.672 People are now, if you like, engaging in the 00:15:59.672 --> 00:16:03.000 more critical mindset, a critical thinking faculty. 00:16:03.000 --> 00:16:07.500 I fully agree. I think education can be one key to solving this. 00:16:07.600 --> 00:16:10.350 But you've mentioned it in your introduction earlier to 00:16:10.350 --> 00:16:13.200 say, OK, there are so many sophisticated tools out there, 00:16:13.400 --> 00:16:16.221 which can be operated at scale so that they are, 00:16:16.221 --> 00:16:19.100 or the adversaries, they are really in a position 00:16:19.400 --> 00:16:23.299 to just throw out that many disinformation and 00:16:23.299 --> 00:16:28.300 misinformation that even this debunking process is too slow 00:16:28.300 --> 00:16:32.100 to deal with the sheer volume of disinformation. 00:16:32.100 --> 00:16:35.700 Isn't that another issue that we are facing when 00:16:35.700 --> 00:16:39.600 we are consuming all these signals on a daily basis? 00:16:41.200 --> 00:16:46.695 Back in my single days, I observed that there 00:16:46.695 --> 00:16:52.800 are a number of ostensibly young, attractive women 00:16:52.800 --> 00:16:57.806 who were contacting me and expressing romantic desire 00:16:57.806 --> 00:17:03.000 and fascination with my somewhat holy, homely features. 00:17:04.200 --> 00:17:07.700 And, you know, they would be of various different nationalities. 00:17:07.800 --> 00:17:17.300 And I think a very common one was the ivory skinned Korean, Japanese, Chinese young lady. 00:17:18.000 --> 00:17:20.200 You mentioned that I'm from Portugal. 00:17:20.200 --> 00:17:23.858 Well, actually, perhaps now if someone was trying to 00:17:23.858 --> 00:17:28.500 romance scam me, they would send a video call of a Portuguese one. 00:17:28.700 --> 00:17:33.200 Maybe I'm more interested in someone who is local as I am happily married. 00:17:33.400 --> 00:17:36.800 So hopefully I am not vulnerable to that particular tick. 00:17:39.800 --> 00:17:43.000 But you bring up another interesting point. 00:17:43.000 --> 00:17:49.706 If it's not one source, if instead you have a multitude 00:17:49.706 --> 00:17:56.900 of orchestrated sources flooding with contradictory claims. 00:17:58.700 --> 00:18:00.700 Chocolate milkshakes can cure COVID. 00:18:00.900 --> 00:18:04.700 No, it's not chocolate milkshakes. You need to drink colloidal silver. 00:18:05.100 --> 00:18:10.400 No, colloidal silver doesn't work. You need to eat grass. 00:18:10.400 --> 00:18:17.200 No, grass doesn't work. You need to drink neat vodka. 00:18:17.500 --> 00:18:21.600 I could see many analysts being quite fond of that, fond of that last one. 00:18:21.700 --> 00:18:30.200 And the goal with the fire hose of falsehood, I call it, is to flood our information zone. 00:18:30.700 --> 00:18:33.193 If you like, the arena which we consume 00:18:33.193 --> 00:18:36.900 information with all these different contradictory claims, 00:18:36.900 --> 00:18:43.400 which exhaust our critical faculty and we give up trying to figure out what's true. 00:18:44.200 --> 00:18:48.500 And, you know, I've done it myself. I really, really don't want to watch the news. 00:18:49.500 --> 00:18:53.900 So we have apathy, confusion, and these are the real objectives. 00:18:54.600 --> 00:19:04.100 So, for example, in Malaysia, I believe there was a very successful campaign that says, 00:19:04.100 --> 00:19:10.800 if you don't support the government, then stay home. Don't vote. That'll show them. 00:19:11.200 --> 00:19:14.434 That's nice. And maybe it really rings a bell 00:19:14.434 --> 00:19:18.100 for somebody. But, of course, the outcome is clear. 00:19:18.700 --> 00:19:21.621 Yeah, the outcome is clear. The activists who are 00:19:21.621 --> 00:19:24.900 voting or interested in the opposition party go hooray. 00:19:25.300 --> 00:19:26.700 Everyone get out to the poll booths. 00:19:27.700 --> 00:19:32.400 So how does this translate for our security, our audience today? 00:19:32.400 --> 00:19:37.400 Well, this is the playbook in some nation state influence operations. 00:19:38.800 --> 00:19:42.453 And if you are a CISO in a large, and even the 00:19:42.453 --> 00:19:46.900 meat, you don't have to be a large organization anymore. 00:19:46.900 --> 00:19:50.161 If you're in the supply chain for a large organization, 00:19:50.161 --> 00:19:53.600 if you're doing something that is a key piece of research, 00:19:55.500 --> 00:20:01.400 you are quite likely to be a target in some nation state influence operations. 00:20:01.400 --> 00:20:05.800 Remembering here, the goal isn't to make us believe the false thing. 00:20:06.600 --> 00:20:09.600 It's to make us uncertain enough to do nothing. 00:20:09.700 --> 00:20:18.900 Do I put in endpoint detection? Do I need to use an MDR? Do I need to upgrade my firewall? 00:20:19.100 --> 00:20:28.000 Do I need to put in place something to prevent agentic pollution? 00:20:28.000 --> 00:20:31.130 We sometimes think about agentic AI. So again, 00:20:31.130 --> 00:20:34.600 the goal isn't to make you believe the false thing. 00:20:34.600 --> 00:20:37.000 It's to make you uncertain enough to do nothing. 00:20:37.900 --> 00:20:40.100 I think that's something we should all think about. 00:20:40.300 --> 00:20:48.100 It's certainly, I think, the reason why places like EIC and in our research companies 00:20:48.100 --> 00:20:57.900 do try to say, here's some clarity in a very vocal and often confused marketplace. 00:20:58.000 --> 00:21:00.881 Let's get back to this later. I just wanted to 00:21:00.881 --> 00:21:03.700 come back to one thing that you've mentioned. 00:21:04.500 --> 00:21:08.737 To say, okay, if you use AI to look more handsome, or if I 00:21:08.737 --> 00:21:12.900 do that, that sounds in the first place a bit ridiculous. 00:21:12.900 --> 00:21:15.413 But if we look at the bigger social media 00:21:15.413 --> 00:21:18.600 platform that especially younger people than us use, 00:21:19.300 --> 00:21:23.100 if you think of Instagram or TikTok, I think there is no 00:21:23.100 --> 00:21:26.900 image of a single person where there is no filter on it. 00:21:26.900 --> 00:21:32.900 And what is a filter other than changing reality, than changing the message? 00:21:33.200 --> 00:21:36.044 And these images, these videos, they are 00:21:36.044 --> 00:21:39.600 picked up by the algorithms and they are promoted. 00:21:39.600 --> 00:21:43.700 And they have the visibility and the reach that is required. 00:21:44.100 --> 00:21:49.300 So do these platforms, think of X, think of Instagram, 00:21:49.300 --> 00:21:54.500 think of TikTok, not even ask for this disinformation? 00:21:55.900 --> 00:22:04.100 Well, as you say, we have to recognize that platform design is not neutral. 00:22:04.100 --> 00:22:11.816 We have to recognize that the algorithms you said for all of these 00:22:11.816 --> 00:22:20.000 different platforms, X, Facebook, TikTok, LinkedIn, Blue Sky, Macedon, 00:22:21.900 --> 00:22:30.400 recommendations optimized for engagement, full stop. Outrage is engaging again, period. 00:22:30.400 --> 00:22:36.000 Which means that that infrastructure is an accelerant. 00:22:36.500 --> 00:22:40.688 So, for example, a music composer who I'm acquainted 00:22:40.688 --> 00:22:45.600 with, I don't know him very well, not as well as I'd like to, 00:22:45.600 --> 00:22:57.900 made a statement which was taken to be supporting very controversial views about autism, 00:22:58.200 --> 00:23:03.200 and was then linked to views regarding gender identity. 00:23:04.100 --> 00:23:09.168 Without again delving into the rights or wrongs of it, 00:23:09.168 --> 00:23:14.800 the original comment was rapidly lost in the roar of outrage 00:23:15.400 --> 00:23:19.800 that occurred across the social media platform. 00:23:20.000 --> 00:23:24.272 With the result that this particular person had to 00:23:24.272 --> 00:23:28.800 retire because it was unacceptable to work with them. 00:23:28.800 --> 00:23:32.900 Not because of what they said, but because of the perception of what they said. 00:23:33.700 --> 00:23:38.835 And so we have a real question here. Is the solution better content 00:23:38.835 --> 00:23:44.200 moderation, or do we actually need to change the incentive structures? 00:23:44.800 --> 00:23:48.500 Do we need to change the optimizing for engagement? 00:23:49.300 --> 00:23:53.212 And that's a much harder problem because clearly all 00:23:53.212 --> 00:23:57.200 of these social media platforms thrive on engagement. 00:23:57.200 --> 00:24:02.460 The more engagement they have, the more they can sell advertising, 00:24:02.460 --> 00:24:07.800 the more they can sell data analysis based on their user platforms. 00:24:08.400 --> 00:24:12.325 And so nobody has solved this and everyone has gone for, well, let's 00:24:12.325 --> 00:24:17.000 optimize for engagement, cross our fingers and hope that the right thing happens. 00:24:17.100 --> 00:24:20.700 There's no evidence to that as yet. 00:24:20.700 --> 00:24:28.544 Is this also related to the erosion of trust in traditional media outlets, 00:24:28.544 --> 00:24:36.600 traditional sources of information that is then replaced by the influencers? 00:24:36.600 --> 00:24:46.000 We live in interesting times and we have our institutional trust. 00:24:46.500 --> 00:24:53.648 So I compare my perception of media, government, even of science, and 00:24:53.648 --> 00:25:00.900 I would say that my institutional trust, how I view the UK government, 00:25:00.900 --> 00:25:06.192 how I view UK media organizations, how I view various different 00:25:06.192 --> 00:25:11.400 places is very different from the way that my parents view it. 00:25:11.800 --> 00:25:15.000 And certainly from the way it was viewed growing up. 00:25:15.400 --> 00:25:20.700 We now don't evaluate information on its merits alone. 00:25:20.900 --> 00:25:24.541 We evaluate the source. And I'm sure many of you listening on 00:25:24.541 --> 00:25:28.600 this podcast will say, well, yes, of course we do. Why wouldn't you? 00:25:28.600 --> 00:25:33.200 If we lose that institutional trust, it creates a vacuum. 00:25:33.900 --> 00:25:38.224 And as we know, nature abhors a vacuum. So it gets 00:25:38.224 --> 00:25:43.500 filled by influencers, by partisan outlets and peer networks. 00:25:43.700 --> 00:25:49.230 And so one of the challenges for all of us in the analyst industry right now is 00:25:49.230 --> 00:25:54.900 that the up and coming generation of technologists tend to rely on peer networks. 00:25:54.900 --> 00:26:00.900 They're more likely to ask a question on Reddit than they are to call up an analyst firm. 00:26:01.200 --> 00:26:05.798 And so we have to change how we are working to make ourselves 00:26:05.798 --> 00:26:11.300 authentic and to make ourselves accountable through different mechanisms, 00:26:11.500 --> 00:26:16.700 because these new modes of communication, influencers, partisan outlets, 00:26:16.700 --> 00:26:21.900 they feel more authentic, but they have fewer accountability mechanisms. 00:26:21.900 --> 00:26:28.600 And so we need to rebuild trust in this new generation of technologists. 00:26:28.900 --> 00:26:33.141 This actually does translate to security leaders who 00:26:33.141 --> 00:26:37.300 are seeing a new generation of employees coming in. 00:26:37.700 --> 00:26:41.079 When your employees don't trust official internal 00:26:41.079 --> 00:26:45.700 communications, they fill the gap with informal channels and rumor. 00:26:45.700 --> 00:26:48.712 Of course, you would say, well, that's the water 00:26:48.712 --> 00:26:52.100 cooler. Gossip has been a thing since time immemorial. 00:26:52.500 --> 00:26:56.500 But it's changed. It is now far more primal. 00:26:57.500 --> 00:27:05.000 Information security depends on a functioning chain of trust. 00:27:06.200 --> 00:27:14.300 And this means that if you have a disinformation attack that breaks that chain of trust, 00:27:14.300 --> 00:27:19.900 if you have disinformation attack that actually tries to replace that chain of trust, 00:27:20.700 --> 00:27:24.366 because our institutional trust in authority is very 00:27:24.366 --> 00:27:29.300 different now, and indeed the next generation will be different again, 00:27:31.500 --> 00:27:33.700 disinformation attacks are quite powerful. 00:27:34.700 --> 00:27:38.450 Trusted messenger problem is one that is going to be a problem 00:27:38.450 --> 00:27:42.200 for all of us in our employment as well as our personal lives. 00:27:42.200 --> 00:27:45.831 I remember many years ago, there was an initiative by 00:27:45.831 --> 00:27:49.600 the musician Peter Gabriel who handed out video cameras 00:27:49.600 --> 00:27:53.208 to make sure that there's proof of things going 00:27:53.208 --> 00:27:57.200 wrong when it comes to states behaving inadequately, 00:27:58.000 --> 00:28:00.700 to policemen behaving inadequately, etc. 00:28:00.800 --> 00:28:04.500 So the video camera and the product of the video camera 00:28:04.500 --> 00:28:08.200 was the actual proof to say there's something going on. 00:28:08.200 --> 00:28:16.300 Today, when we have that good fake information, videos, deepfakes, whatever, 00:28:17.100 --> 00:28:23.700 does that not render the actual evidence not as important, not as trustworthy? 00:28:23.800 --> 00:28:31.600 And maybe is this not even a bounce back from the availability of this deepfake? 00:28:31.600 --> 00:28:35.459 So to say, okay, I do not trust even legitimate 00:28:35.459 --> 00:28:39.400 sources because I don't know if it's legitimate. 00:28:40.300 --> 00:28:43.999 That's very interesting. You mentioned what 00:28:43.999 --> 00:28:48.300 Peter did many years ago, and the action of truth, 00:28:49.100 --> 00:28:56.200 the epitome of truth was video evidence. That's where the phrase came from. 00:28:56.200 --> 00:29:01.200 We are now very aware of deepfakes and AI-generated content. 00:29:02.200 --> 00:29:07.517 And so again, everybody smiled when I mentioned 00:29:07.517 --> 00:29:13.400 the Dalai Lama and me, but clearly I was deepfaking. 00:29:14.300 --> 00:29:18.700 And we now have a paradox, which I would call the liar dividend. 00:29:19.900 --> 00:29:22.800 Real evidence can now be dismissed as fake. 00:29:23.500 --> 00:29:28.059 If you go back to the anecdotes I came in with about the 00:29:28.059 --> 00:29:32.700 General Medical Council trying to regulate practitioners, 00:29:35.100 --> 00:29:39.000 they were saying, well, no, that was a deepfake. 00:29:39.700 --> 00:29:43.422 A politician, when they're caught on video saying 00:29:43.422 --> 00:29:47.600 something damaging, has just claimed it's AI-generated. 00:29:47.600 --> 00:29:52.880 So this mere existence of sophisticated fakes gives bad actors 00:29:52.880 --> 00:29:58.500 plausible deniability for things that in fact are completely real. 00:29:59.700 --> 00:30:05.400 In my view, this is as dangerous as the fakes themselves. 00:30:06.500 --> 00:30:12.100 We don't just believe in the false things, we lose confidence in true ones. 00:30:12.100 --> 00:30:17.800 And this is actually, I think, the most underappreciated aspect of the whole problem. 00:30:19.400 --> 00:30:24.700 I mentioned my parents. They're in their 80s. My mother is a retired medical practitioner. 00:30:26.200 --> 00:30:30.877 And she said to me, I don't know how people of my age cope 00:30:30.877 --> 00:30:36.200 with some of the challenges with the UK's National Health Service. 00:30:38.800 --> 00:30:42.124 I wouldn't be able to manage without my training in 00:30:42.124 --> 00:30:46.100 medicine to make sure that we get the treatment that we need. 00:30:46.500 --> 00:30:50.100 Of course, not everybody of my mother's age is a retired medical practitioner. 00:30:50.400 --> 00:30:53.000 In fact, I'd say they're probably a very small percentage. 00:30:53.700 --> 00:30:58.900 So this liar's dividend is something we all need to be careful of. 00:30:58.900 --> 00:31:05.092 And as security leaders, again, you need to make sure that the information 00:31:05.092 --> 00:31:11.200 you provide, the training and the awareness you provide is unimpeachable. 00:31:11.500 --> 00:31:18.845 Because the moment somebody says, ah, that's just his opinion or her opinion, and 00:31:18.845 --> 00:31:26.100 here's evidence to the contrary, the role of security becomes much, much harder. 00:31:26.100 --> 00:31:29.627 Understood. And in the end, when you translate that into the 00:31:29.627 --> 00:31:33.800 organizations, into the enterprise, into business, that is actual risk. 00:31:34.500 --> 00:31:38.500 But now that we've come full circle, we are back at EIC. 00:31:38.700 --> 00:31:43.500 We need to understand what is out there. And maybe you can describe this a bit. 00:31:43.500 --> 00:31:49.922 But also, how can we make the right steps for preventing our employees, our 00:31:49.922 --> 00:31:56.600 colleagues, fall victim to such a thing and to get better in identifying that? 00:31:56.600 --> 00:31:58.852 Because this is a real risk and it will be 00:31:58.852 --> 00:32:01.800 something that we have to deal for the next years with. 00:32:02.700 --> 00:32:06.400 So, first of all, what are the attack vectors and what can we do? 00:32:07.400 --> 00:32:09.300 Okay. So, yeah, let's look at that. 00:32:09.300 --> 00:32:14.800 I said AI is the accelerant. And by that, I mean that generative 00:32:14.800 --> 00:32:20.300 AI makes it trivially cheap to produce convincing false content. 00:32:20.900 --> 00:32:22.300 I think that's unarguable. 00:32:23.200 --> 00:32:26.800 I said, let's talk about where the enterprise risk is concrete. 00:32:27.000 --> 00:32:30.800 I think we need to do that because I've talked about lots of theoretical things. 00:32:30.900 --> 00:32:35.900 But I'm sure, again, our audience is thinking, well, you know, how does this really work? 00:32:35.900 --> 00:32:38.650 So, let's talk about how we can use the same 00:32:38.650 --> 00:32:41.900 technology for detection and content authentication. 00:32:43.000 --> 00:32:47.200 However, historically in security, offense tends to have the advantage. 00:32:48.200 --> 00:32:52.600 Defenders, that's a lot, are always training on yesterday's attacks. 00:32:53.000 --> 00:32:57.600 So, right now, there are three things that matter for security leaders. 00:32:58.600 --> 00:32:59.300 Voice cloning. 00:32:59.300 --> 00:33:05.100 One, 30 seconds of audio is enough to convince you to clone a voice. 00:33:05.600 --> 00:33:07.700 And by the way, it's free. 00:33:08.300 --> 00:33:09.300 Look at voicebox.sh. 00:33:10.300 --> 00:33:17.000 This can be used for CEO fraud, vishing, and bypassing voice biometrics. 00:33:18.400 --> 00:33:23.600 Anybody using voice biometrics, your authentication system is at risk. 00:33:24.800 --> 00:33:27.000 In fact, I want to actually go further than that. 00:33:27.200 --> 00:33:29.919 If voice authentication is part of your control 00:33:29.919 --> 00:33:33.100 set, part of your authentication stack, it's weakening. 00:33:36.100 --> 00:33:38.500 Secondly, deepfake video calls. 00:33:39.600 --> 00:33:44.700 We have recent evidence of an employee at a company called Arup 00:33:45.700 --> 00:33:54.100 wired $25 million after a live deepfake video call impersonating the CFO. 00:33:54.900 --> 00:33:55.900 Not a phishing link. 00:33:56.800 --> 00:33:59.900 A meeting with a face and a voice. 00:34:00.400 --> 00:34:05.200 I talked about, you know, we've seen this in romance scams, but hey, here we are. 00:34:05.400 --> 00:34:06.600 It's right in our workplace. 00:34:07.100 --> 00:34:13.700 Our human instinct is to trust what we see and hear. 00:34:13.700 --> 00:34:16.000 And that's a vulnerability. 00:34:17.100 --> 00:34:22.400 The third, and for those of you who work in fraud, you'll have heard this phrase. 00:34:23.300 --> 00:34:24.200 Synthetic identity. 00:34:25.700 --> 00:34:31.100 By this, I mean AI generated people that don't exist. 00:34:31.800 --> 00:34:37.900 They're used for account creation fraud, fake employee references, vendor impersonation. 00:34:37.900 --> 00:34:42.200 The know your customer tools and onboarding 00:34:42.200 --> 00:34:47.200 controls were frankly not designed for this world. 00:34:48.100 --> 00:34:50.000 And they are outmatched. 00:34:51.300 --> 00:34:55.200 So I said, let's talk about what does work and what we can do. 00:34:55.400 --> 00:34:58.700 Well, clearly don't rely on technical detection. 00:34:59.200 --> 00:35:02.700 And I would say the generation side is moving too fast. 00:35:03.100 --> 00:35:05.400 You need to rely on process controls. 00:35:05.400 --> 00:35:11.000 We need to rely on out-of-band verification for any high-value request. 00:35:12.600 --> 00:35:18.100 So if somebody gives you a video call saying, I'd like you to wire $25 million, 00:35:19.400 --> 00:35:21.800 that to me would be a high-value request. 00:35:22.000 --> 00:35:24.960 And so it's worth you calling the CFO on their 00:35:24.960 --> 00:35:28.500 landline, on their mobile, whatever you have available. 00:35:28.500 --> 00:35:31.694 But out-of-band of the video call and saying, 00:35:31.694 --> 00:35:35.100 hey, did you really want me to wire $25 million? 00:35:36.900 --> 00:35:39.781 Regardless of who appears on the call, even if 00:35:39.781 --> 00:35:42.600 it's a CFO with a reputation of being grumpy, 00:35:43.200 --> 00:35:47.800 if it's a CEO who's got the reputation of firing people who don't comply with his demands. 00:35:49.700 --> 00:35:54.800 It is important, and it's important in our security awareness training, 00:35:54.800 --> 00:36:02.200 that we empower people with the ability to question what they are being presented with. 00:36:02.900 --> 00:36:05.500 Again, dual authorization for wire transfers. 00:36:06.500 --> 00:36:13.300 It's actually much harder to scam two people than it is one. 00:36:13.600 --> 00:36:16.600 There's a whole host of psychological reasons why this is. 00:36:17.000 --> 00:36:21.600 But nevertheless, if you are trying to scam me to send a wire transfer, 00:36:21.600 --> 00:36:26.400 it's going to be much harder if you have to scam Matthias as well. 00:36:27.700 --> 00:36:31.400 Code word communications. Again, this is a shared secret. 00:36:31.600 --> 00:36:33.300 We know that shared secrets are vulnerable. 00:36:33.600 --> 00:36:37.400 But they can help in sensitive, objective communications. 00:36:38.100 --> 00:36:42.100 So all of these tools can help us defend. 00:36:42.400 --> 00:36:47.600 But the main thing, the most important thing, and I must reemphasize this again, 00:36:47.600 --> 00:36:54.200 train people on the pattern of the attack before they encounter it. 00:36:54.600 --> 00:36:54.900 Inoculate. 00:36:55.000 --> 00:36:58.100 Right. So it's the same thing that we started with. 00:36:58.300 --> 00:37:01.400 You mentioned the awareness training that we are doing right now 00:37:01.400 --> 00:37:06.400 when trying to identify phishing mechanisms and checking links in emails. 00:37:07.100 --> 00:37:12.600 It's just taking one step back and also not only trusting links, 00:37:12.600 --> 00:37:17.700 but also not trusting faces, voices, even if they look convincing, 00:37:18.000 --> 00:37:20.900 just to make sure that this could be easily faked. 00:37:21.000 --> 00:37:24.700 This is the message to convey to our colleagues, to our family, to our friends? 00:37:24.900 --> 00:37:29.400 I think so. And I think if I was to leave with a closing thought, 00:37:30.700 --> 00:37:35.300 we've spent 20 years teaching people not to trust unexpected emails. 00:37:36.400 --> 00:37:42.600 But I'm not proud to say that when we recently changed our security training provider, 00:37:43.100 --> 00:37:49.400 I fell foul of the first spoof email they sent me. 00:37:49.700 --> 00:37:53.800 So if I can fall for it, frankly, anybody can fall for it. 00:37:53.800 --> 00:37:59.000 You just need to be sufficiently stressed and you need to be under pressure. 00:37:59.400 --> 00:38:00.600 And there you go. 00:38:01.600 --> 00:38:07.500 But we now need to teach people not to trust unexpected voices and faces. 00:38:08.000 --> 00:38:13.400 That's harder because trust in human perception is deeply wired. 00:38:14.300 --> 00:38:18.100 We are biologically inclined to believe what we see and hear. 00:38:18.800 --> 00:38:24.400 And so we need to make sure that one of those at the organization that does adapt 00:38:24.400 --> 00:38:30.800 and adapt fast, we need to have new process controls, new identity verification 00:38:31.100 --> 00:38:35.300 and the cultural awareness to question a face on a screen. 00:38:36.100 --> 00:38:37.800 That will be the real advantage. 00:38:37.900 --> 00:38:43.300 And this also demands for some bravery to say, OK, I dare to do so. 00:38:43.300 --> 00:38:48.900 I do not trust that this is my CEO, that this is my colleague that I've met last year, 00:38:48.900 --> 00:38:51.500 but he's in the department in the US. 00:38:51.500 --> 00:38:58.300 So it really demands more standing by those who actually do it. 00:38:58.400 --> 00:39:00.800 And I think that is maybe also part of the story to tell. 00:39:02.000 --> 00:39:02.300 There is. 00:39:02.200 --> 00:39:07.600 I mean, there's been some psychological research done on the different personality types 00:39:07.900 --> 00:39:14.800 in organizations, people who are rule makers, rule shakers, rule followers, rule blockers. 00:39:14.800 --> 00:39:21.800 And we need to, again, tune the message for each of the different personality types. 00:39:22.500 --> 00:39:30.400 But yes, ultimately, it's the CEO, the CFO, all the senior executive leadership 00:39:30.400 --> 00:39:34.300 of a company to say you are empowered to question. 00:39:34.500 --> 00:39:38.900 If it's me, you will not be fired for questioning this. 00:39:38.900 --> 00:39:40.900 In fact, you could even change it. 00:39:41.300 --> 00:39:50.600 We will reward people who successfully challenge a deepfake. 00:39:51.300 --> 00:39:57.700 So I think there's a lot of motivational tools we can use here. 00:39:57.700 --> 00:40:03.100 But the most important thing is to say people need to feel it's OK to ask questions. 00:40:03.200 --> 00:40:04.400 It's OK to challenge. 00:40:04.400 --> 00:40:08.400 A new type of second factor being built into the brains of our colleagues. 00:40:08.800 --> 00:40:13.600 So to really double check whether this is something that is really OK. 00:40:14.100 --> 00:40:15.900 So thank you very much, Jonathan. 00:40:15.900 --> 00:40:19.800 That was really interesting and really went full circle. 00:40:20.000 --> 00:40:23.650 We went from TikTok to filters to your parents 00:40:23.650 --> 00:40:27.300 up until to how can we protect our enterprise. 00:40:27.300 --> 00:40:30.457 And I think that is really the full scope of this 00:40:30.457 --> 00:40:34.000 topic of misinformation slash disinformation and trust. 00:40:34.200 --> 00:40:36.500 We will follow up on that at EIC. 00:40:37.200 --> 00:40:41.969 Of course, it is baked into the overall fabric of 00:40:41.969 --> 00:40:48.100 identity because this is where attackers are trying to fool us. 00:40:48.700 --> 00:40:50.700 Any final words before we close down? 00:40:51.300 --> 00:40:52.700 Just, yeah, I echo that. 00:40:52.700 --> 00:40:57.800 I know there are some vendors who are really working hard to challenge this. 00:40:58.500 --> 00:41:01.200 And I know that many of them are going to be at EIC. 00:41:01.900 --> 00:41:09.300 I look forward to talking with all of you and say workshopping, discussing this problem. 00:41:09.500 --> 00:41:14.400 And maybe we come up with some innovative solutions to this in this area. 00:41:14.700 --> 00:41:15.100 Thank you. 00:41:15.300 --> 00:41:20.700 And for the audience, if you have questions, please leave them below this video on YouTube 00:41:20.700 --> 00:41:22.900 or send Jonathan or me a mail. 00:41:23.300 --> 00:41:26.617 If you have any suggestions how to work even more in this 00:41:26.617 --> 00:41:30.400 topic and to have another episode on that going into more detail, 00:41:30.800 --> 00:41:31.400 let us know. 00:41:31.800 --> 00:41:32.600 Send us your questions. 00:41:33.100 --> 00:41:38.700 If you want to talk to Jonathan just about this topic or to me, please reach out to us. 00:41:38.800 --> 00:41:40.400 We are happy to do so. 00:41:41.200 --> 00:41:44.000 And for the time being, thank you very much, Jonathan. 00:41:44.300 --> 00:41:45.200 Thank you for your time. 00:41:45.300 --> 00:41:48.500 This has been a longer episode, but it was worth every second. 00:41:48.500 --> 00:41:53.500 And I'm really looking forward to having you soon on this podcast and see you next time. 00:41:53.700 --> 00:41:54.200 Thank you, Jonathan.