Getting a Grip on Your AI: Know What It Does. Understand the Risks and Rewards. Be Compliant

Hello everyone. And welcome to this webinar to discuss getting a grip on your AI. Remember that midst all the hype, there are concrete wins for your company. If applied correctly. My name is Annie Bailey and I am an Analyst with your coal. So your call delivers a variety of services centering on identity and access cybersecurity and artificial intelligence. So this comes in the form of reports, webinars like this advisory projects and events. In terms of reports, we have also a wide variety of forms that deliver different value to you. So I encourage you to explore the different report types we have on KC plus, and this is our new content and research platform. So I encourage you to go to our website and check it out.
We also offer advisory services and this offers a roadmap of the different assistance we can provide to your company. We also have events and one event I might point out to you is the AI impact summit, which is on November 20 through 20 and Munich. So I encourage you. If you're interested in the content of this webinar, you will certainly find more interesting ideas and discussions there. So to talk about some important things for this webinar, audio control, you are muted centrally. And so there is no need to worry about this at all. We have handled that this webinar is being recorded and you will have access to the recording very shortly, as well as the slide deck. These will be available for you to download and also for questions, we are taking questions at the end. And so feel free to submit your question at the go to webinar control panel. So the agenda for our webinar today will take us through some current key challenges that you should be aware of when looking to integrate AI solutions. We set the stage by talking about their survivorship bias, which is a helpful anecdote. Then we'll continue by discussing two aspects that need to be built into upcoming AI systems, robust AI. We also highlight two tendencies to when integrating an AI solution, data bias and data hoarding.
So to achieve a helpful frame of mind, I'll begin by reminding you of the survivorship bias. It takes its name from a historical scenario. During a war, planes were often shot down and pilots were lost. Engineers puzzled long and hard over how to strengthen the plane's frame in order to reduce the number of pilots they lost. And so they studied the planes that returned carefully noted the areas that showed damage and reinforced those areas. But to no avail, there was no improvement in how many planes and pilots survived, but you've probably already spotted their mistake. There was no need to reinforce sections of the returning planes that showed damage. If there were able to fly home with that damage, it was probably strong enough to do the job. Instead, the engineers needed to identify areas on the returning planes that never had damage. If no plane were managed to return home with damage to that particular area, that was the critical weakness that engineers needed to reinforce. So the survivorship bias is this. When you only consider the survivors as a representative sample, you miss critical insight into the population as a whole,
We should keep the survivorship bias in mind when considering AI solutions, we've heard of the big failures like faulty resume sorting algorithms or self-driving vehicle accidents. These are the planes that didn't return. We have a benefit that the engineers didn't have. We can see that the planes didn't make it home and diagnose what went wrong, but we can still see this shadow of those big failures and the planes that do return home, the AI systems that seemingly work well. So our goal should be to observe the full spectrum of successful and failed projects that are being implemented across different industries. And to remember that a representative sample will provide more accurate insight that only looking at the characteristics of successful projects observe the aspects of those solutions where no question, no controversy has yet been raised. It's possible that the spotless sections of an AI solution are the critical weaknesses of those that fail. So this is a useful exercise in your own organizations, identify what main skills your AI solution should have, and look at the successful and failed projects with that. Learning from others development with the survivorship bias in mind will help identify the critical weaknesses of that particular skill, target, those critical weaknesses with your investment and R and D funds successful aspects. That skill can be purchased. There's no need to reinvent the wheel. So allocate your resources towards reinforcing the aspects that have not yet been standardized.
So let's continue on with robust AI. We can consider the AI system itself now and what aspects need to be built into it. Robustness being one of those robustness is the ability of a computer system to cope with erroneous input and complications. During execution, AI solutions are designed to process huge amounts of data so that we don't have to with so much input data, we have to expect that it will en encounter incorrect data and familiar data or even adversarial attacks. This is why robustness should be a top priority in AI programming, adding robustness to an AI tool. Isn't a superficial, additional feature. Robustness should be integrated into its function and to its design. There are strategies for building robust. So physical systems and embodiment need high level execution monitoring, which observes the environment to determine the need for a new plan. This new plan may be necessary because of missing preconditions effects or state variable variables where essentially the AI agent is given an incorrect model of the world. Execution monitoring can be broken into three steps, action monitoring, where the AI agent verifies that all the preconditions still apply. Like the wheels are still in working order.
Then there's plan monitoring where the AI agent verifies that the remaining plan will succeed such as the way to reach the destination is accessible with wheels and find legal monitoring where the AI agent checks to see if there is a better goal than the one it was assigned such as that the destination is still the most appropriate out of all the options. Execution monitoring is an iterative process to achieve robustness, not an explicit loop of defined actions, instead of only trying one repair pan, one's a repair plan, excuse me, the AI agent should rotate through a range of possible repairs and use it as a learning opportunity. Robustness learns from its failures and constantly improves its functionality. When dealing with the software side, such as machine learning, there are different methods of robustness testing, machine learning and deep learning have made huge contributions to the accuracy and range of solutions available in image classification, speech recognition, language translation, and many other applications. But these systems can be compromised by carefully crafting and feeding adversarial inputs to a trained model with the intent producing an incorrect output. And an example with image, classifiers can be that a house is classified as a dog because of maliciously inserted noise to the input image.
Targeted adversarial testing is a way to defend against such malicious attacks. Adversarial testing essentially defines the worst case scenario or input for machine learning model, which then helps the model learn how to cope with images that it cannot trust this training scheme, augments the training set with corrected images. This system is then trained to do its actual job like differentiate between dogs and houses. But the hope is that as this method develops, the AI system will be able to recognize input that have been tampered with, and that it should not trust adversarial attacks. Of course do not end when you have a strong in place, there will be new waves of compromising your system, unfortunately. So you have to continually monitor your system. This should sound a little familiar simply because an AI system can successfully navigate or handle a training data set doesn't mean that it will be successful. The AI system that successfully passes a test, it has been trained to do will not have bullet holes in it. When you observe the planes that don't return, don't look only at where they've been hit, but also at where they haven't create opportunities to find the problematic and build reinforcement to the survivorship.
There
Are many methods that are being researched to build robustness in AI machine learning systems. Distillation is one of those distillation transfers training against adversarial attacks between neural networks, a generative adversarial network, or GA N is also being discussed by researchers as an option to defend against adversarial attacks. There are many more strategies for adding robustness to a machine learning system, but currently each strategy can only address specific type of adversarial attack. The task of creating a general defense for multiple types of attacks is still under development by researchers. And the major criticism of these options is that it creates a trade off between robustness and performance of the AI model. The more robust a system is against adversarial attacks. The less efficient the model is during friendly operations,
But as you've probably noticed, robustness is more of a mindset than integrating a one size fits. All solution testing for robustness requires a unique perspective. We are, we are testing machine intelligence with human intellect, but trying to assess the reactions of a non-human intellect requires a non-human test, making different assumptions about how it will react than if you were observing a human actor. The robustness test has to reflect the different neural networks and come from the perspective of an AI agent maintenance is also an ongoing goal, which contributes to robustness without the competence and reliable knowledge that all parts are in top shape. The AI agent will have to overcome its own issues of poor performance before achieving its goal. Robustness includes human element as well, such as having a robust team multidisciplinary teams, which can indicate contextual, legal, social, physical weaknesses, and an algorithm all add to its ability to overcome unfamiliar data in situations. And there is a problem of being too robust. If a program won't terminate, when it is clearly not going to function, the AI system should recognize this and terminate its action safely when necessary include a strong termination analysis in your system development.
So now let's move on to keeping AI explainable people understand the algorithms use for machine learning because people as an AI programmers wrote them. But once an AI system has been trained, it's not clear what information from the input data makes them reach their decision. When we cannot explain why an AI agent reaches a decision it's as if the critical information that enables transparency, accountability, and responsibility for actions or locked away in a black box, it is unexplainable how or why, ah, it is unacceptable. How or why a system arrived at a certain output aside from the very general statement that it was trained to do. So that is unacceptable for the mission and safety critical solutions that are being designed in the medical field for autonomous vehicles or security purposes. Accountability is a major force in the current business environment and any AI system that is adopted should be able to provide your company with insight into why a decision was reached. Robustness. As we discussed earlier is an attempt to avoid mistakes so that there will not be a need to explain erroneous action, but RO robustness will not be enough if there is a mishap in AI system deployment. When questions of accountability must be answered.
But a question which may come to your mind is if you really need to implement AI, that is potentially unexplainable. What has been termed white box AI models do exist. They rely mostly on decision trees and linear regressions that deliver standard statistical insights. The appeal for black box AI that is powered by machine learning is that it delivers predictive insights with increasing accuracy. So accuracy shouldn't be the single highest priority. We just discussed that there is a trade off between robustness performance. Being able to explain your AI models. Decision is one aspect of robustness. The idea that accuracy cannot be the highest priority is a difficult one to get away from, especially when dealing with seemingly simple harmless tasks, such such classification. One example is of an AI system that was tasked with separating the images of Huskies and of wolves. It was highly accurate in its training data set, but the original algorithm didn't have any capability to explain why it could tell the difference between Huskies and wolves. When an explanation algorithm was applied retrospectively, it became clear that the critical input in the photos to differentiate Huskies and wolves was the presence of snow in the background.
This shows that accuracy can be achieved at the expense of knowledge. The AI system, the AI system learned a shortcut, but couldn't actually distinguish a difference in the way the animals looked. This illustrates the range of issues that could arise in other applications, such as in nature, natural language processing for job resume sorting or determining the chance of criminal re-offend AI systems that have tried to accomplish these goals have used racial and gender in the puts as a shortcut when making their decisions. And although they could have given accurate predictions, given today's gender and racial power imbalances ethically, these shouldn't be their primary decision indicators. There is significant research going into making such AI systems trained with machine learning, more explainable. Some of this focuses on applying retrospective analysis to an already trained system to understand what input it prioritizes,
Ah, excuse me.
In our image classification example, this can be thought of as applying a heat map to the image pixels, which highlight which pixels the algorithm decided were most relevant. So you can see an example of this on the slide above where the original picture is on the left column and the heat maps to different varieties have been applied in the center column and the column on the right. So this style can also be used in text analysis by requesting a relevant score for each word, which was pro processed by the AI system solutions like these are being developed and they should be requested when implementing an AI system in your own solutions, but they are not perfect yet. And they require additional analysis from the human counterparts. Another solution is to insert requests for explanation, for decisions, and a much earlier phase in the development learning process. Instead of applying an explanation algorithm retrospectively, as we just discussed, some researchers are wondering if explainable AI shouldn't be inherent to the system even while it is in its learning phase. This can be thought of as rewarding an AI system that decides based on human values that have been defined for the program or building feedback into each decision,
Achieving explainable AI in this way requires a different process for machine learning. That includes an ethics training aspect. So next we continue reducing data bias, bias, and data exists. So how do we, as flawed and biased individuals remove our unconscious non-politically correct assumptions from the data that we give to machine learning stocks. And how do we prepare for the inevitable future when our current actions are biased or bigoted, according to the values of the future,
Transparency
Is the first step. There must be a record of how data scientists interacted with the input data such as inquiries analysis. And if the data was ever tampered with this at least keeps the door open to explain how or why an algorithm learns the way it did was trained the way it was this mics. It possible to explain the whole system, which is not as fine grained as explaining each decision, but it is still necessary. This allows for observation by third party, data scientists and specific feedback for improvement, reducing data bias should be done with a multi-stakeholder approach, including multi-gender multicultural and multi-disciplinary applying human values and decision making will help reduce harm to vulnerable people. Groups pre-processed training data with group fairness and individual distortion constraints consider the unintended consequences of loss of opportunity, economic loss, social detriment, or loss of Liberty. A multi stakeholder team comes with its own challenges such as agreeing on a cohesive vision of the word fair eliminating social bias relies on a understanding of what people groups and its complexities. One sec such question could be the fair ratio of female CEOs. This could be interpreted as AC as the accurate current number of female CEOs, which is of course different in different countries,
Or it could be determined to be the estimate of what can be achieved in the future, like a 50, 50 male to female ratio as an aspirational measurement of what is fair. So this must be an active and iterative conversation to continually identify and eliminate bias. Social bias receives the most attention in the press because it's an easy story to communicate, but thankfully AI systems are not producing biased outcomes for social reasons. They rely on the numerical inputs. So it's not as if these seemingly racist systems develop the social prejudice when official recognition model incorrectly identifies minority groups, the systems are revealing the biases of the data they were trained on today. The scientists, this is a known evil dealing with statistical bias is standard practice in data analysis. So there should be hope in being able to mitigate data bias in AI models. And there are vendors who are developing tools, such as bias, bias rating systems to measure and increase the fairness. So mitigating bias is a huge issue, and I hesitate to leave you with such a positive outlook for reducing bias and data, because it ultimately relies on reducing bias in humans, which has been an age-old struggle. The more awareness that is built for such issues, the more equitable AI systems can become.
But there in lies the deepest level of complexity, which is who OFS has the power to determine what is equitable and fair. This is a challenge that all AI systems will be subject to in the coming years. So side note, but perhaps just as important is to address the behavior that naturally accompanies the AI hype, which is data hoarding. It is natural to assume that the more data you can collect, the better that it will lead. You lead you to more insights and efficiencies that AI systems could potentially unlock for you. But holding on to every scrap of data you have is not LA. It is not strategic, nor will it, will it help much if or when that unimagined AI system is implemented? The reality is that only certain data will be advantageous to you in the future when your ideal AI systems are implemented, keep data that makes sense, work together with an interdisciplinary team in your organization who understands the challenges and compliance issues surrounding data storage, but be sure to include the visionaries who can imagine the ideal bespoke AI solutions of the future. This will help narrow down the type of data to set aside for future use and avoid the mess of too much in applicable data.
You can also mitigate your risks. Storing large amounts of data comes with its own challenges such as cleaning and managing the data so that it is actually usable in the future, but a larger threat than simply this organization is attracting hackers with such a large collection of data comes the high chance of being the target of an attack. So avoid creating a honey pot of data, which attracts the attention of malicious attackers compliance should also remain a high priority. Despite the temptation of saving all potentially useful data. The PII data that is collected should be limited to the minimum data required to perform a service. So we came across a few recurring themes in this webinar that you should be sure to take with you avoiding the survivorship bias requires you to look at the aspects of a problem that appear strong to you. Question how your perspective influences your view robustness and explainability should be fully part of any AI solution achieving robustness and explainable. AI is a question that is unique to each different AI application. It is necessary to monitor the developments of this research, which is advancing quickly. And although there is a long tradition of checking for statistical bias, bias is a human problem, and this will have to be an iterative and reflective process to eliminate bias before it creates negative impacts in an AI model's performance and in society at large, and remember data hoarding, it is attempting, but an avoidable behavior. And lastly, considering all of these points, a multi-disciplinary team has clear advantages. It is one way of helping you see the situation from multiple angles and imagine being the pilot under fire instead of the engineer safe in his workshop.
So I thank you for your participation and I invite you to submit a question if you have any I'll a few, if anybody would like to, and if there are no questions, then I want to, once again, thank you for your participation. I invite you to be familiar with Kohl's other research and knowledge platforms, which are available, who reports through advisory and of course, events. And I look forward to seeing you at a future event or at another webinar. So thank you very much.