WEBVTT 00:00:00.000 --> 00:00:20.600 Welcome to the KuppingerCole Analysts Chat. I'm your host. My name is Matthias Reinwarth. 00:00:20.600 --> 00:00:27.000 I'm analyst and advisor with KuppingerCole Analysts. Our guest today is Alexei Balaganski, 00:00:27.000 --> 00:00:32.400 and we want to talk about a topic we have not yet covered. But first of all, hi Alexei, 00:00:32.400 --> 00:00:33.400 good to have you. 00:00:33.400 --> 00:00:37.000 Hello Matthias, and great to be back again. 00:00:37.000 --> 00:00:42.200 Great to have you. We want to talk about a rather large topic and we want to scratch 00:00:42.200 --> 00:00:49.700 the surface for the first time. We want to talk about AI data provenance and AI data 00:00:49.700 --> 00:00:52.162 management in general. It's difficult to 00:00:52.162 --> 00:00:55.300 start. There's maybe a brief definition of what you 00:00:55.400 --> 00:01:00.600 consider to be AI data provenance and everything that's related from your perspective. 00:01:01.200 --> 00:01:07.400 Right. Well, Matthias, first of all, you are right. It's kind of a new, barely explored 00:01:07.400 --> 00:01:12.600 topic for not just KuppingerCole, but for everybody, I guess, including our listeners, 00:01:12.600 --> 00:01:18.500 simply because on the one hand, it's so obvious, but on the other hand, nobody is actually 00:01:18.500 --> 00:01:20.976 talking much about it until, I guess, it will be 00:01:20.976 --> 00:01:23.400 too late because we will be hit by some kind of 00:01:24.200 --> 00:01:30.100 regulation. And yeah, one of the biggest challenges we had in living in the AI age is 00:01:31.100 --> 00:01:38.400 all those huge piles of AI slop, if you will. Low quality generated data that's obviously 00:01:39.400 --> 00:01:44.800 unusable and not true. The results of hallucinations and poor data decisions and 00:01:44.800 --> 00:01:48.600 low quality inputs, you know, garbage in, garbage out. That's what we get 00:01:48.900 --> 00:01:52.204 a lot. And I mean, if you are watching a 00:01:52.204 --> 00:01:56.500 poorly made, generated video on YouTube showing some 00:01:57.200 --> 00:02:02.300 hitters playing music instruments, I guess it's still okay. But if the same level of data 00:02:02.300 --> 00:02:04.808 provenance and quality management applies to 00:02:04.808 --> 00:02:07.600 your business decisions, you are in huge trouble. 00:02:07.600 --> 00:02:13.000 This is exactly what we want to discuss and how to prepare for this and how to avoid all 00:02:13.000 --> 00:02:17.500 the consequences in the future. Right. So we all have learned the hard way, 00:02:18.400 --> 00:02:22.177 that providing good data as the foundation for 00:02:22.177 --> 00:02:26.200 good results of AI is essential. But on the other 00:02:26.200 --> 00:02:29.606 hand, this usage of the data and the overall 00:02:29.606 --> 00:02:33.400 topic of our topic today, the AI data provenance, 00:02:33.400 --> 00:02:39.400 is becoming an urgent issue. What exactly do you mean when you say that we have to expect 00:02:39.400 --> 00:02:45.200 that crisis, that we might get hit by the regulation, as you said? Where does it start 00:02:45.200 --> 00:02:51.200 and where can we see that as indication? Right. Well, it all started basically when 00:02:51.200 --> 00:02:58.000 all those AI giants like OpenAI and other companies have started training their models. 00:02:58.000 --> 00:03:00.480 They needed huge amounts of data for 00:03:00.480 --> 00:03:04.200 training. And basically, what they decided to do is to 00:03:04.200 --> 00:03:10.000 scrape the entire internet with complete disregard to copyright and other regulations, 00:03:10.000 --> 00:03:17.000 without asking permissions from anyone. They just basically have taken the entirety of the 00:03:17.000 --> 00:03:19.817 data generated by humans and somehow 00:03:19.817 --> 00:03:24.200 accessible through the internet and used all the data to 00:03:24.200 --> 00:03:29.700 train their models. I guess it worked to an extent. So we have all those wonders of modern 00:03:31.000 --> 00:03:36.200 chatbots and AI assistants. Unfortunately, it's now very difficult to differentiate 00:03:37.100 --> 00:03:43.400 whether this response was triggered by some training data from a reputable expert or a 00:03:43.400 --> 00:03:46.276 copyright owner or it was something from a 00:03:46.276 --> 00:03:49.700 Reddit post. You remember all those stories like a 00:03:49.700 --> 00:03:56.300 chatbot recommending you to put glue on your pizza or to misidentify poisonous mushroom or 00:03:56.300 --> 00:03:59.278 anything like that. So again, garbage in, 00:03:59.278 --> 00:04:03.200 garbage out. If your data model is trained entirely on 00:04:03.500 --> 00:04:09.000 Wikipedia and Reddit, well, you cannot expect any sufficiently useful results. And again, 00:04:09.000 --> 00:04:12.150 while it still kind of works for consumer-grade 00:04:12.150 --> 00:04:15.300 decisions, it would definitely catastrophically 00:04:15.300 --> 00:04:17.524 fail when you start applying it for critical 00:04:17.524 --> 00:04:20.000 business decisions or for critical infrastructure 00:04:20.000 --> 00:04:26.500 management. Even if there is no existing regulations for this yet, sooner or later, 00:04:26.500 --> 00:04:29.179 it will appear simply because, well, we cannot 00:04:29.179 --> 00:04:31.800 leave like that. We have to do something. And 00:04:31.800 --> 00:04:37.500 again, at least when it comes to copyright, obviously, this has already started. Movie 00:04:37.500 --> 00:04:40.950 studios, artists, singers, you name it, all the 00:04:40.950 --> 00:04:44.400 content creators who actually invested a lot of 00:04:44.400 --> 00:04:48.606 the real human effort and money into the content 00:04:48.606 --> 00:04:52.900 are now complaining because all those things were 00:04:52.900 --> 00:04:55.101 used without their permission and without any 00:04:55.101 --> 00:04:57.400 compensation. So this is probably the first and 00:04:57.400 --> 00:04:59.600 the most important driver behind those future 00:04:59.600 --> 00:05:01.800 regulations, but we will definitely see more. 00:05:01.800 --> 00:05:05.116 Right. And if I think just briefly out of the 00:05:05.116 --> 00:05:08.800 IAM context and the auditors coming in and saying, 00:05:08.800 --> 00:05:11.223 okay, why does Matthias have this access right? 00:05:11.223 --> 00:05:13.800 You need to have proper auditing, a complete trail 00:05:13.800 --> 00:05:16.392 of why this access has been assigned, and you 00:05:16.392 --> 00:05:19.100 need to prove that. So this is logging, this is 00:05:19.100 --> 00:05:21.971 documenting what has happened. If an AI does such 00:05:21.971 --> 00:05:24.900 a decision that Matthias should or should not have 00:05:24.900 --> 00:05:29.406 an access right, it gets blurred already to say, 00:05:29.406 --> 00:05:34.100 okay, will this decision be correct? On what is it 00:05:34.100 --> 00:05:37.413 based? And if we get to complete AI-generated 00:05:37.413 --> 00:05:40.800 content, then it gets much more fuzzy and much 00:05:40.800 --> 00:05:43.550 more interesting when auditors come into play. 00:05:43.550 --> 00:05:46.300 So it's much more than traditional logging and 00:05:46.300 --> 00:05:48.600 auditing, right? Right. Well, I guess you are 00:05:48.600 --> 00:05:50.900 completely right. You are hitting the nail on 00:05:50.900 --> 00:05:54.709 its head here. It is actually, on the one hand, 00:05:54.709 --> 00:05:58.600 this is nothing new. It is based on some kind of 00:05:59.100 --> 00:06:01.810 audit trail for everything. So obviously this 00:06:01.810 --> 00:06:04.700 is not something you have to invent from scratch 00:06:04.700 --> 00:06:08.236 just for AI purposes. You already have tons 00:06:08.236 --> 00:06:12.100 of the data as application logs, database logs, 00:06:12.600 --> 00:06:18.200 just basically traces of everything which is happening within your business, both business 00:06:18.200 --> 00:06:21.531 related and IT related, if you will, or identity 00:06:21.531 --> 00:06:25.000 related. The only problem is that it's not enough. 00:06:25.000 --> 00:06:28.084 First of all, all the data is scattered across 00:06:28.084 --> 00:06:31.100 different silos. Second, as soon as AI models 00:06:31.100 --> 00:06:34.549 come into play, you have this kind of media 00:06:34.549 --> 00:06:38.400 break. You know that AI models are by definition 00:06:38.400 --> 00:06:41.276 non-deterministic because they never actually 00:06:41.276 --> 00:06:44.600 deal with the original training data. They deal with 00:06:45.500 --> 00:06:49.429 weights, tokens, hashes, some basically random 00:06:49.429 --> 00:06:53.700 numbers which were derived from the original data. 00:06:54.200 --> 00:07:02.100 And often this process is completely blind and proprietary. So it's very difficult to 00:07:02.100 --> 00:07:06.100 recreate that path back into the past reliably. 00:07:06.100 --> 00:07:10.100 So yes, you have to think in more than one way. 00:07:10.700 --> 00:07:17.900 You have to have all that historical original data readily available and prepared for AI 00:07:17.900 --> 00:07:21.611 consumption. But you also have to have some kind 00:07:21.611 --> 00:07:25.400 of a race and explainability trail for the actual 00:07:25.400 --> 00:07:28.370 model reasoning. And of course, when you actually 00:07:28.370 --> 00:07:31.400 have an architecture like REG, where you reach out 00:07:31.400 --> 00:07:34.532 to third party data, you have to have the data 00:07:34.532 --> 00:07:37.800 traced as well, both the actual data and all the 00:07:37.800 --> 00:07:40.818 operations you have performed. So it's not new. 00:07:40.818 --> 00:07:43.900 There is no need to reinvent the wheel. You just 00:07:43.900 --> 00:07:47.035 have to juggle a lot of different wheels to make 00:07:47.035 --> 00:07:50.300 it work. And this is perhaps the biggest challenge 00:07:50.300 --> 00:07:53.544 for the actual business. Right. And if you use 00:07:53.544 --> 00:07:57.000 your own data, your own business data as training 00:07:57.000 --> 00:08:00.000 material, I think, as you said, that can be well 00:08:00.000 --> 00:08:03.000 documented, the lineage of data, how you use it, 00:08:03.800 --> 00:08:07.148 how it was used to train the LLM. But if you 00:08:07.148 --> 00:08:10.800 consider everything that is within your favorite 00:08:10.800 --> 00:08:18.700 AI provider's model, if you consider this as your supply chain in the end, then it really 00:08:18.700 --> 00:08:21.557 becomes tricky. And I think many organizations 00:08:21.557 --> 00:08:24.600 are not yet really well prepared to understanding 00:08:24.600 --> 00:08:27.580 that there needs to be auditability, traceability 00:08:27.580 --> 00:08:30.500 in the end, a proper way of dealing with, again, 00:08:30.500 --> 00:08:34.270 back to AI data governance. So this is really a 00:08:34.270 --> 00:08:38.200 challenge for many organizations where they just, 00:08:38.200 --> 00:08:42.300 where many just resign to say, OK, I can't do that. Can they? 00:08:43.600 --> 00:08:47.459 Well, absolutely. I mean, the data lineage as a 00:08:47.459 --> 00:08:51.400 domain thing, like a problem is not new. I mean, 00:08:51.400 --> 00:08:54.078 we have been doing this for decades. And in some 00:08:54.078 --> 00:08:56.700 industries, it has been a mandatory requirement 00:08:56.700 --> 00:09:00.075 anyway, for decades as well. When it comes to 00:09:00.075 --> 00:09:03.600 AI, you're right. It is a massive chunk of your 00:09:03.600 --> 00:09:06.700 software supply chain nowadays. And unfortunately, 00:09:06.700 --> 00:09:09.800 we do not have an S-bone. We don't have a software 00:09:09.800 --> 00:09:12.315 bill of materials for AI models yet. And 00:09:12.315 --> 00:09:15.900 definitely not for AI data. Well, yeah, at the very best, 00:09:15.900 --> 00:09:19.032 you can trust the AI vendor claims that they were 00:09:19.032 --> 00:09:22.100 not using Reddit to train that model. But how do 00:09:22.100 --> 00:09:26.596 you verify that? Do you actually get some kind of 00:09:26.596 --> 00:09:31.000 a step-by-step explanation of the model thinking 00:09:31.000 --> 00:09:34.183 process showing that, yes, it was definitely 00:09:34.183 --> 00:09:37.800 not touching Reddit or Wikipedia or someone else's 00:09:37.800 --> 00:09:44.900 unproven data and so on and so forth? No, not yet. But this will definitely be a critical 00:09:44.900 --> 00:09:50.300 requirement in the future, because again, this is what differentiates AI from AI slop. 00:09:50.900 --> 00:09:54.023 Right. And I think these are also different 00:09:54.023 --> 00:09:57.800 types of expectations when it comes to what you see. 00:09:57.800 --> 00:10:00.409 But when I use an AI, I want to create a 00:10:00.409 --> 00:10:03.800 proper set of results that meet my expectations. And 00:10:03.800 --> 00:10:06.301 that is quality, that is style, that is 00:10:06.301 --> 00:10:09.700 overall applicability for my given task. But auditors 00:10:09.700 --> 00:10:12.219 or somebody who wants to make sure that 00:10:12.219 --> 00:10:15.900 everything has been processed the right way based on data 00:10:15.900 --> 00:10:21.800 that is well-documented, they have a different focus and there is a clash between the two. 00:10:22.500 --> 00:10:26.600 Can that be synchronized? Can that be made work in parallel? 00:10:27.200 --> 00:10:30.837 Well, yes and no, because this is what we usually 00:10:30.837 --> 00:10:34.400 believe, that the auditor is out there to punish 00:10:34.400 --> 00:10:37.953 us for not doing the paperwork right, but not 00:10:37.953 --> 00:10:41.900 really. All that paperwork, I mean, it was created 00:10:41.900 --> 00:10:47.900 because of a long trail of blood lost by businesses of not following that paperwork 00:10:47.900 --> 00:10:51.134 in the past. So actually, the compliance auditor 00:10:51.134 --> 00:10:54.300 is on your side, because they want to make sure 00:10:54.300 --> 00:10:57.483 that you are doing your job properly. Because if 00:10:57.483 --> 00:11:00.600 you won't, well, you will have many problems in 00:11:00.600 --> 00:11:02.635 the future, which are not related to 00:11:02.635 --> 00:11:05.800 compliance at all. It will be security issues or quality 00:11:05.800 --> 00:11:08.718 issues, obviously, legal repercussions if you 00:11:08.718 --> 00:11:11.700 fail to, for example, respect copyright owners 00:11:11.700 --> 00:11:16.298 and so on. So yes, it's up to you to ensure that 00:11:16.298 --> 00:11:20.800 your work is done properly, that all your feed, 00:11:20.800 --> 00:11:24.800 all the data that you feed into your AI models 00:11:24.800 --> 00:11:28.800 has proper provenance and lineage and quality, 00:11:28.800 --> 00:11:31.574 and you have the means to prove it. The only 00:11:31.574 --> 00:11:34.600 thing you have to do specifically for compliance 00:11:34.600 --> 00:11:37.107 is basically follow best practices and 00:11:37.107 --> 00:11:41.000 regulations and prescribed formats and protocols and so on. 00:11:41.000 --> 00:11:47.200 But again, this is secondary. You have to be vigilant. You have to follow all this AI data 00:11:47.200 --> 00:11:50.211 provenance things by default, because if you 00:11:50.211 --> 00:11:53.700 won't, you will fail spectacularly sooner or later. 00:11:53.700 --> 00:11:58.700 The quality of your digital product will suffer. Again, your legal standing and your brand 00:11:58.700 --> 00:12:05.100 reputation will suffer. And again, the compliance fine would be just icing on the cake. 00:12:05.700 --> 00:12:08.295 So you should never think about compliance as a 00:12:08.295 --> 00:12:11.000 means to an end. It's just the way to ensure that 00:12:11.000 --> 00:12:14.400 you are doing the rest of your job properly. 00:12:14.400 --> 00:12:17.350 Right. And the fact that we're talking about it right 00:12:17.350 --> 00:12:20.300 now, of course, the disclaimer, we are not lawmakers, 00:12:20.300 --> 00:12:22.278 we are not lawyers, we're not auditors. But in 00:12:22.278 --> 00:12:24.300 the end, why are we talking about it right now? 00:12:24.300 --> 00:12:32.300 What has changed? Can we see that in real life, that the times of the Wild West and AI 00:12:32.300 --> 00:12:37.700 at least partially are over, so that we need to do things properly? What has changed? 00:12:38.600 --> 00:12:41.318 Well, again, we are definitely not lawyers, but we 00:12:41.318 --> 00:12:44.200 say that lawyers are actually the first driving force 00:12:44.200 --> 00:12:49.500 behind this. Again, as I mentioned earlier, copyright is probably the most sensible driver 00:12:49.500 --> 00:12:52.488 behind it, because, again, you can easily move 00:12:52.488 --> 00:12:55.800 and demonstrate that the current AI implementations 00:12:55.800 --> 00:12:59.139 are disregarding copyright laws on a huge and 00:12:59.139 --> 00:13:02.700 massive scale. It kind of used to work for a few 00:13:02.700 --> 00:13:09.300 years. And this is how companies behind all those huge and successful AI models actually 00:13:10.400 --> 00:13:13.368 managed to amass all that knowledge. But sooner 00:13:13.368 --> 00:13:16.400 or later, we are coming to the realization that, 00:13:16.400 --> 00:13:20.200 no, it will no longer work. You have to do something, you have to compensate 00:13:21.600 --> 00:13:23.754 the efforts of the original content creators, 00:13:23.754 --> 00:13:26.100 you have to follow the existing legal frameworks. 00:13:26.100 --> 00:13:28.551 So, yes, legal people and regulations are 00:13:28.551 --> 00:13:31.600 behind the first push. But, of course, they are not 00:13:31.600 --> 00:13:34.634 the only ones. We are, I mean, we're keeping a 00:13:34.634 --> 00:13:37.800 call, we are operating within the cyber security 00:13:37.800 --> 00:13:41.098 and identity markets, and we see a lot of 00:13:41.098 --> 00:13:45.200 similar push as well. If you are making security or 00:13:45.200 --> 00:13:48.824 identity decisions on the data, which you cannot 00:13:48.824 --> 00:13:52.600 prove is even remotely relevant and good enough to 00:13:52.600 --> 00:13:55.192 make those decisions, those decisions will be 00:13:55.192 --> 00:13:57.900 garbage. And which would mean that you will not 00:13:57.900 --> 00:14:01.133 block the data breach, you will let the malicious 00:14:01.133 --> 00:14:04.300 actor in on the stolen credential, and so on and 00:14:04.300 --> 00:14:11.400 so forth. The rule is universal. Garbage in, garbage out. And nowhere it applies more than 00:14:11.400 --> 00:14:17.100 in cyber security. Because, again, a wrong decision, a single wrong decision is enough to 00:14:17.100 --> 00:14:22.600 open your entire digital infrastructure to a ransomware attack or a massive data breach. 00:14:22.600 --> 00:14:27.400 And it will be too late to fix. This is not something you can kind of backtrack. 00:14:28.200 --> 00:14:34.600 This is not something you can investigate with forensics. Too late. You have to prepare in 00:14:34.600 --> 00:14:38.800 advance. You have to make sure that all your decisions are made on proper data. 00:14:39.500 --> 00:14:43.400 Same story, same issue, same AI data provenance. 00:14:43.400 --> 00:14:46.777 Right. And when organizations are doing things 00:14:46.777 --> 00:14:50.300 properly and want to do things properly, how can 00:14:50.300 --> 00:14:54.150 you add governance to existing systems? When we 00:14:54.150 --> 00:14:58.000 are all using AI, we are providing a chatbot as 00:14:58.000 --> 00:15:00.872 keeping a call. Many others are doing this as 00:15:00.872 --> 00:15:04.000 well. We are using that, our own data as training 00:15:04.000 --> 00:15:06.968 material. But how to add that proper level of 00:15:06.968 --> 00:15:10.200 governance to that? Run a system for half a year, 00:15:10.200 --> 00:15:15.100 then I say, okay, I need to add some proper governance to allow for this data provenance. 00:15:15.100 --> 00:15:19.400 Is this possible in retrospect? Well, I mean, at least it's the 00:15:19.400 --> 00:15:23.400 billion-dollar question, if you will, because it's definitely not something that we can 00:15:23.400 --> 00:15:25.824 properly answer in five minutes or even half an 00:15:25.824 --> 00:15:28.300 hour. This is something that the entire industry 00:15:28.300 --> 00:15:31.700 and the entire humankind needs to decide on a 00:15:31.700 --> 00:15:35.100 global scale, and it will probably take years 00:15:35.100 --> 00:15:38.071 to figure out properly. But one thing which is 00:15:38.071 --> 00:15:41.300 clear is that you cannot bolt it on in retrospect, 00:15:41.800 --> 00:15:47.700 on the one hand. You have to actually ensure that everything follows the same principles. 00:15:47.700 --> 00:15:55.400 All your data, all your third-party data, all your internal and third-party AI models, 00:15:56.000 --> 00:15:59.437 and of course, all the data flows between those 00:15:59.437 --> 00:16:02.800 endpoints, they all have to be covered by this 00:16:02.800 --> 00:16:08.400 governance framework. It is extremely difficult. Yes, this is true. But on the other hand, 00:16:09.500 --> 00:16:12.123 again, you don't have to reinvent everything from 00:16:12.123 --> 00:16:14.800 scratch. Parts of this are surely already covered, 00:16:14.800 --> 00:16:16.837 especially if you are in a regulated industry 00:16:16.837 --> 00:16:19.100 already, like if you are a financial organization, 00:16:19.100 --> 00:16:21.789 a bank, you already have a lot of governance 00:16:21.789 --> 00:16:24.600 controls for your own data. Somehow we have to 00:16:25.200 --> 00:16:28.662 force your primary third-party suppliers to do 00:16:28.662 --> 00:16:32.200 the same. Again, for AI models, it is difficult 00:16:32.200 --> 00:16:38.000 because we are very early on the stage of developing those formats and regulations, 00:16:38.000 --> 00:16:43.100 but this is definitely coming soon. There are some developments out there already. 00:16:43.800 --> 00:16:45.733 You remember we've talked about AI 00:16:45.733 --> 00:16:49.200 explainability for years. This is actually one of the primary 00:16:49.200 --> 00:16:54.600 use cases for explainability because explainability means traceability and auditability. 00:16:55.400 --> 00:16:58.796 If you can explain how exactly this decision 00:16:58.796 --> 00:17:02.500 was made, you are already halfway on showing the 00:17:02.500 --> 00:17:07.000 auditor that you are doing it properly. And if you cannot, well, you failed already. 00:17:07.800 --> 00:17:13.000 So, yeah, this is difficult. This is very fragmented. This is the same. This is like 00:17:13.000 --> 00:17:17.700 an extension of your software supply chain security, if you will, which is a huge topic 00:17:17.800 --> 00:17:21.200 on its own and which is a topic which we are covering exactly now as well. 00:17:22.700 --> 00:17:27.800 Exciting times, tons of upcoming research, but again, we are not lost, you are not lost, 00:17:27.800 --> 00:17:30.933 we are in it together. So, let's work together. 00:17:30.933 --> 00:17:34.200 Right, and I think you've mentioned both aspects, 00:17:34.200 --> 00:17:38.200 explainability on the one hand and non-deterministic systems on the other hand, 00:17:38.200 --> 00:17:45.400 which of course produces this inner conflict of achieving a full level of governance, of 00:17:46.000 --> 00:17:52.400 achieving the right level of explainability. But nevertheless, we can start to make things 00:17:52.400 --> 00:17:59.300 better or properly. What would be practical approaches to say, okay, this is the minimum 00:17:59.300 --> 00:18:04.300 to start with, documenting your data catalog, documenting your data, having the data still 00:18:04.300 --> 00:18:07.332 available for reference purposes, would that be 00:18:07.332 --> 00:18:10.300 a good starting point? Well, obviously, if you 00:18:10.300 --> 00:18:12.781 cannot tell even to yourself where your data 00:18:12.781 --> 00:18:15.600 is, well, you are already failed. You have already 00:18:15.600 --> 00:18:21.800 failed even before the auditors come. So, yes, this is absolutely a foundational crucial 00:18:21.800 --> 00:18:24.569 requirement. You have to know where your data 00:18:24.569 --> 00:18:27.400 is. So, I guess you start as usual with almost 00:18:27.400 --> 00:18:30.031 every kind of security strategy. You start 00:18:30.031 --> 00:18:33.100 with your posture management. You have to catalog 00:18:33.100 --> 00:18:35.578 everything and not just once, but all the 00:18:35.578 --> 00:18:38.600 time. That's exactly the point. That's exactly the 00:18:38.600 --> 00:18:45.200 difference of dealing with non-deterministic decision chains. You have to basically keep 00:18:45.200 --> 00:18:48.101 traces of everything, of every decision, 00:18:48.101 --> 00:18:51.800 because you cannot expect that when you are redoing 00:18:51.800 --> 00:18:54.200 the same steps again, that they will produce the 00:18:54.200 --> 00:18:56.600 same result. It will never happen just by nature 00:18:56.600 --> 00:19:00.114 of the AI. So, either the model, the actual 00:19:00.114 --> 00:19:04.200 decision maker has to have this tracing, auditing, 00:19:04.900 --> 00:19:10.900 protocoling capabilities built in. Maybe this will happen sometime. Or you have to build a 00:19:10.900 --> 00:19:15.400 universal extension of your existing data governance solution to cover that 00:19:16.500 --> 00:19:19.931 quote-unquote attack surface as well. So, somebody 00:19:19.931 --> 00:19:23.500 has to do it. There is no single way yet, but again, 00:19:23.500 --> 00:19:29.700 the industry is evolving. What we observe, for example, among the data security suppliers, 00:19:29.800 --> 00:19:34.600 that they are absolutely thinking about this already. And they are already working on some 00:19:35.900 --> 00:19:42.000 solutions. I can only recommend reading our existing research on data security platforms, 00:19:42.000 --> 00:19:48.400 for example, and on data quality and lineage we've done earlier. All those things 00:19:49.800 --> 00:19:54.000 plug in together, at least they should. And of course, some specialized 00:19:55.000 --> 00:19:58.400 generative AI defense tools, if you will, for the lack of better terms, 00:19:58.400 --> 00:20:04.900 they've come into play as well. They are very early in their kind of maturity and 00:20:04.900 --> 00:20:07.729 productization stages, but they are emerging. And 00:20:07.729 --> 00:20:10.500 we're also doing some research on those tools as 00:20:10.500 --> 00:20:17.000 well. They all somehow have to come together in another kind of a fabric, if you will. 00:20:17.500 --> 00:20:20.583 Kuping et al. has defined identity fabrics and 00:20:20.583 --> 00:20:23.800 security fabrics, and everybody is talking about 00:20:23.800 --> 00:20:27.590 data fabrics now. Well, this is your AI fabric, 00:20:27.590 --> 00:20:31.300 AI data fabric, AI provenance fabric, whatever 00:20:31.300 --> 00:20:35.068 you want to, but you have to know what's going 00:20:35.068 --> 00:20:39.000 on. You have to trace every AI decision from the 00:20:39.000 --> 00:20:42.840 actual start, from the raw data you had supplied 00:20:42.840 --> 00:20:46.600 for training to the actual operational data you 00:20:46.600 --> 00:20:52.800 have supplied as context, to every step of the inference process, and finally, to all the 00:20:53.800 --> 00:20:56.328 activities which were made, let's say, by an 00:20:56.328 --> 00:20:59.200 agent based on that decision. Everything has to be 00:20:59.800 --> 00:21:04.112 tracked, traced, audited, and somehow combined to 00:21:04.112 --> 00:21:08.600 be able to provide a single protocol of everything. 00:21:09.400 --> 00:21:14.600 Do they have tools like that already? Probably not, but they will have to, because again, 00:21:14.600 --> 00:21:17.450 this is what's definitely coming in the future, 00:21:17.450 --> 00:21:20.300 sooner or later. AI data provenance regulations 00:21:20.300 --> 00:21:22.649 will be the next GDPR, if you will. Exactly. I 00:21:22.649 --> 00:21:25.100 wanted to get to that quote exactly, because you 00:21:25.100 --> 00:21:27.527 said we were all struggling with GDPR 00:21:27.527 --> 00:21:31.200 compliance, because there is no compliance, there's only 00:21:31.200 --> 00:21:33.205 incompliance. If you fail, then you can prove 00:21:33.205 --> 00:21:35.300 it, but you cannot prove that you're compliant, 00:21:35.300 --> 00:21:38.181 which is always an issue. So it's always like a 00:21:38.181 --> 00:21:41.000 security issue or a security incident. As long 00:21:41.000 --> 00:21:43.273 as you think you're secure, nothing has happened, 00:21:43.273 --> 00:21:45.500 all fine. If there's an incident, okay, then you 00:21:45.500 --> 00:21:50.100 have just failed. Maybe also just as a side note 00:21:50.100 --> 00:21:54.700 or a thought, if we think back to the high times 00:21:54.700 --> 00:21:57.983 of big data, I think many of the principles 00:21:57.983 --> 00:22:01.800 that had to apply back then need to apply to AI as 00:22:01.800 --> 00:22:04.679 well. So if you combine data of different origins, 00:22:04.679 --> 00:22:07.500 of different lineage, you need to understand what 00:22:07.500 --> 00:22:10.892 the result is, what the product, the report 00:22:10.892 --> 00:22:14.600 is, and how you apply proper access control and 00:22:14.600 --> 00:22:20.400 management to that as well. With AI, this is the same principle on steroids, much faster, 00:22:20.400 --> 00:22:23.789 much more agile, but the same principle behind 00:22:23.789 --> 00:22:27.400 that. So if provenance is the new GDPR readiness, 00:22:27.900 --> 00:22:30.097 what should executives, those who are 00:22:30.097 --> 00:22:33.600 responsible, those who will most probably get punished when 00:22:33.600 --> 00:22:36.430 something fails, what do they do in the next 12 00:22:36.430 --> 00:22:39.500 months? Well, first of all, of course, they have to 00:22:39.500 --> 00:22:42.682 start thinking about it. Because if they don't do 00:22:42.682 --> 00:22:45.800 it today, they will be woken up in the middle of 00:22:45.800 --> 00:22:48.324 the night sometime next year and told that something 00:22:48.324 --> 00:22:50.800 really bad happened. So they have to start thinking 00:22:50.800 --> 00:22:54.386 about it today. But at the same time, I would urge 00:22:54.386 --> 00:22:57.900 them not to make too hasty decisions. Again, just 00:22:57.900 --> 00:23:03.400 like with quote-unquote AI security a couple of years ago, there will be a lot of 00:23:04.700 --> 00:23:06.881 snake oil peddlers, if you will, who will tell 00:23:06.881 --> 00:23:09.300 you that just buy our product and you will have all 00:23:09.300 --> 00:23:12.487 this figured out for you. No, this is not how 00:23:12.487 --> 00:23:16.100 it's supposed to work. Primarily because nobody can 00:23:16.100 --> 00:23:22.600 decide for you your priorities, your risk environments and risk habitats. You have to make 00:23:22.600 --> 00:23:25.766 decisions, you have to prioritize what data 00:23:25.766 --> 00:23:29.300 you want to capture and trace and protect first, 00:23:30.000 --> 00:23:32.585 because it's your business decision. And also 00:23:32.585 --> 00:23:35.400 you have to remember that at least parts of those 00:23:35.400 --> 00:23:37.800 requirements and future regulations are 00:23:37.800 --> 00:23:41.000 probably already addressed by existing tools. Again, 00:23:41.000 --> 00:23:46.400 if you have a database, you probably already have some data security controls in place. 00:23:47.000 --> 00:23:50.166 And you don't have to replace them with new AI 00:23:50.166 --> 00:23:53.400 aware ones. You just have to probably make sure 00:23:53.400 --> 00:23:56.600 that they support an upcoming logging format or 00:23:56.600 --> 00:23:59.800 something like that, or a connector to feed the 00:23:59.800 --> 00:24:08.000 data into a third party processing application. Watch this space, understand the strategic 00:24:08.000 --> 00:24:13.900 decisions, definitely look for interesting developments and announcements. Come visit our 00:24:13.900 --> 00:24:16.324 conferences, for example, because this is where 00:24:16.324 --> 00:24:18.800 we discuss all those things. Watch our webinars. 00:24:18.800 --> 00:24:20.878 But again, kind of before you make a purchasing 00:24:20.878 --> 00:24:23.000 decision, make sure that you understand what you 00:24:23.000 --> 00:24:25.886 are actually getting. Look behind the labels, 00:24:25.886 --> 00:24:28.900 never trust, always verify. That's kind of zero 00:24:28.900 --> 00:24:31.744 trust for AI data provenance, if you will. 00:24:31.744 --> 00:24:35.400 Should apply everywhere. But again, your strategic aim 00:24:35.400 --> 00:24:39.658 should be not just compliance by default, but 00:24:39.658 --> 00:24:44.200 AI data provenance by default. If you know where 00:24:44.200 --> 00:24:47.500 all your data is coming from and what's happening 00:24:47.500 --> 00:24:50.800 to all your data, then you already have just like 00:24:50.800 --> 00:24:53.321 90 percent of everything figured out, 00:24:53.321 --> 00:24:57.000 regardless of a specific use case. That should be your 00:24:57.500 --> 00:25:00.777 ideal to strive for. Right. And we're talking 00:25:00.777 --> 00:25:04.200 about an AI that works as it is supposed to be. 00:25:04.200 --> 00:25:07.167 We have current research ongoing that will be 00:25:07.167 --> 00:25:10.200 published soon, which looks into LLM security, 00:25:10.200 --> 00:25:17.100 protecting the AI from external malicious influences of any kind. That, of course, 00:25:17.100 --> 00:25:19.500 needs to be a priority as well, to make sure that 00:25:19.500 --> 00:25:21.900 the AI works as you expect it to be and that it's 00:25:21.900 --> 00:25:25.146 not influenced by bad actors from outside. This 00:25:25.146 --> 00:25:28.600 is an additional aspect that we just did not cover 00:25:28.700 --> 00:25:32.063 in that context, but of course has influences 00:25:32.063 --> 00:25:35.500 on how the data is actually processed and even 00:25:35.500 --> 00:25:38.826 extracted or even additional data ingested at 00:25:38.826 --> 00:25:42.300 runtime. So prompt injection is a thing that we 00:25:42.300 --> 00:25:45.155 should talk about in another episode, right? 00:25:45.155 --> 00:25:48.400 But again, let me just emphasize one takeaway from 00:25:48.400 --> 00:25:51.512 this. Do not focus on point solutions. Do not 00:25:51.512 --> 00:25:54.900 think that prompt injection is the root of all AI 00:25:54.900 --> 00:26:02.500 evils. No, it's just one fairly well-researched attack vector. And again, if, for example, 00:26:02.500 --> 00:26:10.100 all your data is always encrypted, if all your LLM interfaces are always monitored, 00:26:10.100 --> 00:26:13.132 and you have some kind of a guardrail installed 00:26:13.132 --> 00:26:16.100 in place, then who cares that prompt injection 00:26:16.100 --> 00:26:18.950 will be handled for you? Because even if they 00:26:18.950 --> 00:26:21.800 are actually able to steal some of your data, 00:26:21.800 --> 00:26:27.400 it will still be encrypted. Or if they actually manage to overcome some internal 00:26:28.200 --> 00:26:31.400 LLM guardrail, they will still not be able to exfiltrate the data. 00:26:32.200 --> 00:26:35.994 You have to think in terms of doing everything 00:26:35.994 --> 00:26:40.200 properly across all your environments, not just AI, 00:26:40.200 --> 00:26:43.763 all your data, all your data flows, ideally even 00:26:43.763 --> 00:26:47.400 in use. If you can manage that, at least for your 00:26:48.200 --> 00:26:51.686 highest and the most critical systems, then it's 00:26:51.686 --> 00:26:55.100 like a multi-factor authentication. They say it 00:26:55.100 --> 00:27:02.400 will automatically negate 95% of all the identity-based attacks. I guess you can find the 00:27:02.400 --> 00:27:05.787 same level of controls for your data as well. 00:27:05.787 --> 00:27:09.400 And again, it's not just about tools, it's about 00:27:09.400 --> 00:27:14.200 processes and workflows. If you have them figured out and configured properly, 00:27:15.100 --> 00:27:18.148 you are automatically solving a huge chunk of 00:27:18.148 --> 00:27:21.400 your potential security and compliance problems. 00:27:21.400 --> 00:27:24.290 So always think strategically, always think from 00:27:24.290 --> 00:27:27.300 the other end, if you will. Not how to please your 00:27:27.300 --> 00:27:30.633 auditor or how to fight another ransomware 00:27:30.633 --> 00:27:34.600 attack. How to make sure you are resilient to both 00:27:35.800 --> 00:27:40.200 attacks and audits. This is the best way. Prevention is the best medicine. 00:27:41.100 --> 00:27:43.569 Right. And as you've mentioned already, we are 00:27:43.569 --> 00:27:46.200 all in this together. We are learning while we're 00:27:46.200 --> 00:27:49.557 working. And so this is really maybe one of the 00:27:49.557 --> 00:27:53.200 biggest takeaways for me to say, okay, yep, we need 00:27:53.200 --> 00:27:56.600 to start doing things properly. When AI is part 00:27:56.600 --> 00:28:00.000 of our delivery process of our products, of our 00:28:00.000 --> 00:28:02.886 services, we need to deal with it as we would 00:28:02.886 --> 00:28:05.900 do with any other component that we provide. So 00:28:06.000 --> 00:28:08.950 provide the proper scrutiny, provide the proper 00:28:08.950 --> 00:28:11.900 governance to that. But nevertheless, this is a 00:28:11.900 --> 00:28:15.941 problem or a challenge at scale. Any final advice 00:28:15.941 --> 00:28:19.900 for enterprises that just feel overwhelmed, just 00:28:19.900 --> 00:28:23.200 over-challenged by this task? Well, again, this 00:28:23.200 --> 00:28:26.500 is a billion dollar question. It might actually 00:28:26.500 --> 00:28:29.561 boil down all those recommendations and 00:28:29.561 --> 00:28:33.800 strategies into a single takeaway. We probably know we 00:28:33.800 --> 00:28:36.816 cannot. This is not something which we can do 00:28:36.816 --> 00:28:40.100 immediately. But again, one thing is to not think 00:28:40.100 --> 00:28:43.976 you can sit through it and it will boil off and 00:28:43.976 --> 00:28:48.100 become another blockchain or something that nobody 00:28:48.100 --> 00:28:51.148 will carry in five years. No, this will never 00:28:51.148 --> 00:28:54.400 happen. Even if AI hype will end up with a crash 00:28:55.200 --> 00:29:01.200 and some large industry leaders will probably cease to exist in five years, it's not going 00:29:01.200 --> 00:29:04.029 anywhere. We are still a digital society. We are 00:29:04.029 --> 00:29:06.800 still digital businesses. We still have to deal 00:29:06.800 --> 00:29:10.100 with huge challenges. And to a large extent, 00:29:10.100 --> 00:29:13.700 there will always be some kind of a digital data 00:29:14.300 --> 00:29:17.450 supply chain problem. And all these things we 00:29:17.450 --> 00:29:20.600 have discussed now, they apply to any kind of 00:29:20.600 --> 00:29:24.757 those issues, future or current. AI is just one. 00:29:24.757 --> 00:29:29.000 Yeah, maybe a very popular, but just one use case 00:29:29.000 --> 00:29:32.915 of that. Again, strategically, do not think in 00:29:32.915 --> 00:29:37.000 labels or buzzwords. Think in risk, if you will, 00:29:37.000 --> 00:29:39.611 because this is what matters in the end. 00:29:39.611 --> 00:29:43.200 Exactly. And so all the truisms hold true. Start small, 00:29:43.200 --> 00:29:46.334 start at all, apply a risk-based approach, and 00:29:46.334 --> 00:29:49.400 really solve the problems that are closest to 00:29:49.400 --> 00:29:51.672 your business first, and then continue 00:29:51.672 --> 00:29:54.900 learning from others. And learning from others, you've 00:29:54.900 --> 00:29:57.507 mentioned our events, our research. We will 00:29:57.507 --> 00:30:00.600 cover this, obviously, as this is part of security, 00:30:00.600 --> 00:30:07.000 and identity management plays an important role. So this is all a part of the solution, 00:30:07.000 --> 00:30:10.642 and we are here to support you. And Copenhagen 00:30:10.642 --> 00:30:14.600 will look at that market in much more detail soon. 00:30:15.100 --> 00:30:19.500 Of course, I need to recommend the European Identity and Cloud Conference as one of the 00:30:19.500 --> 00:30:23.539 key cybersecurity events with a strong focus on IAM, 00:30:23.539 --> 00:30:27.500 but beyond. And our research is essential, I think. 00:30:27.500 --> 00:30:29.850 Thank you very much, Alexei, for being my guest 00:30:29.850 --> 00:30:32.200 today. This will not go away, and we will cover 00:30:32.200 --> 00:30:35.070 this topic in more detail and with different 00:30:35.070 --> 00:30:38.200 angles, and looking also really at more of these 00:30:38.200 --> 00:30:40.850 point solutions sometimes when it comes to really 00:30:40.850 --> 00:30:43.500 implementing things properly. For the time being, 00:30:43.500 --> 00:30:46.056 thank you, Alexei. Well, it was a pleasure, as 00:30:46.056 --> 00:30:49.000 usual, being in these discussions with you, Matthias. 00:30:49.000 --> 00:30:53.000 Thanks a lot, and have a nice day. Have a nice day. Bye-bye.