Cyber Defense Matrix

Hi, I am Sun ou. I live in Washington DC and oh, this is torn.

I, I'm currently CTO for a company focused on solving some interesting challenges in ai. But before that, I used to be the chief scientist at Bank of America. And when I was there, I had a couple different roles. I was a mad scientist, I was a evaluator of different products. I also ran teams that tried to break into the bank. And in the process, I had a chance to really understand this landscape that we call cybersecurity. And it's a very challenging landscape for those who are maybe struggling to understand the breadth of all things in cybersecurity.

Well, that's what I had to do. My challenge was trying to figure out how to make sense of all these things that we see in cybersecurity all the time. And my goal was to find gaps. So let me ask you, do you see any gaps?

Now, you might see gaps, but it's probably hard to find because what you see is all there is. And so I said I need a structure. I need something that's more structured to help me identify where gaps are in all these things that you see here. And so I came up with a simple structure. And that simple structure base is based on two things. Things that I care about and things that I do. The things that I care about are nouns. The things that I do, of course are verbs. And so these five things that I care about, and five things that I do align nicely into what I call the cyber defense matrix.

So that's what you have in front of you, the Cyber Defense Matrix. Now, with the SI Cyber Defense Matrix, I can take all those buzzwords that we saw previously and now organize them more functionally. And when I start to do that, well, gaps become a lot more apparent. Okay? So of course I can do this with product categories, but I could also do it with vendors too. So you'll see a lot of vendors outside, right? Do you know what they sell? Or more importantly, do you know where they, what they sell, how they fit into your program? And if you don't, it's partly because they try to confuse you.

Okay? Now, part of what we'll do is to figure out where things like this map, and that's really what that workshop is all about. Okay?

Now, there's a person by the name of George Box, and he had this wonderful quote, which is, all models are wrong. So this model that I'm showing you is absolutely wrong, okay? Because it won't solve every problem that you have in cybersecurity, but it has turned out to be extraordinarily useful. So his quote is, all models are wrong, but some are useful. And what I have found with the cyber defense matrix is that it's been very useful for a lot of things. And so when I first presented this in the US back in 2016, I came up with a lot of use cases.

Now, my original use case was just mapping vendors, but I also mapped a lot of other things. And these other things end up, you know, some of them worked and some of them didn't quite work. But nonetheless, it turned out that as an organizational system, the cyber defense matrix was really useful to make sense of the world. And there's a lot of use cases.

Now, I wrote about some of these use cases in the book, but we're actually gonna go through some of these use cases as a workshop as well. Okay?

Now, one, one analogy I use to help us understand this process is food. And so we of course, all understand food 'cause we have to eat it every day. And the interesting thing about the cyber defense matrix is we can use it as a framework to map a lot of different types of categories, things that are very common to us in the cybersecurity practice. So for example, what do we have in our current set of capabilities? What's in our cupboard or pantry? What are recipes? Things that we have to, we have to comply with. What are compliance requirements? What are architectural frameworks?

What are good practices? We also want to be able to understand our nutritional needs. These are things like threats, like ransomware. How does ransomware fit into the matrix? We wanna understand what's available in the grocery store, in the supermarket. So we walk outside, we see a vendor, where do they fit in the matrix? And then lastly, we are also concerned about allergies. Allergies are like business constraints, and we'll talk about that more as well. Okay? So ultimately we want to take each of these and we can map them to the matrix.

And so, as an example, let's look at recipes. So let the CIS center for internet security, they have a set of 18 controls. Here's how they map into the matrix, all the different controls, and you can see the rough distribution of where every one of them are. You also see gaps too, right?

Well, the CIS controls, they're just 18 controls. Where's number 19? Where's number 20? Are they even covering some of these other ones that we see that we don't have any that we have gaps in? But anyway, that's it. Here's an example of recipes and where those fit. Then we have nutritional needs. We have threats like fishing and ransomware. So as an example, this is from a friend of mine, Ross Young. He has identified a bunch of different types of threats and the different types of controls that you need for each of the different types of threats.

And if we were to look at ransomware specifically, here are the boxes that are threatened when it comes to ransomware. So if you want to have coverage for ransomware, if you want ransomware controls, then these are the boxes that you need to have covered now. But you can see how by layering different, the different parts of the matrix, you can see how this becomes more interesting, because I could take these areas of threats, add in the recipes, then add in what's available in the supermarket, and then I can highlight those particular areas, nutritional needs, threats that are more prevalent.

So I could say this particular box here, I have a, a higher degree of concern. And then lastly, I wanna capture allergies. So how do we want to, I wanna make sure that we account for the fact that not all these boxes can be covered. And that's because the business will say, that will cause business impact for me.

If you put something in, okay, if you try to implement a security control and you've had no problems whatsoever in implementing that control across your business, then you, I, you, you probably don't actually have a business because I, I haven't run into a situation where the business doesn't push back and say, well, hold on, let's make sure this makes sense in the context of our business. Well, it turns out there are whether there's different profiles for different types of business units.

So if you've ever, for example, one of the basic controls that we have in CIS is removing administrator rights, okay? Removing administrator rights from your, from your, your endpoints.

Well, let me ask you, have you ever tried to remove administrator rights from developers? Okay, that laugh tells you the story here, which is you end up with an allergy, the developers say, this will prevent me from doing any work. So I cannot have that control in here. So there's a allergies that developers have, have we tried to remove? Have we tried to tell sales and marketing? We need to pre-approve every app, every application that you use before you use it, okay? Doesn't work because they wanna try every new application under the sun.

And if you, if you have to approve, approve it every time, then it just slows them down. Now, there's, there's balances here that we need to trade off some balances that we, we need to work to, but we have to understand that putting in certain security policies and constraints or security controls creates problems with the business. And so we have to recognize those are constraints. And so with those constraints, we then can layer on those on top of the matrix as well.

And so now you have this com more complete view of what are your options are, what controls you need to meet, what threats you're trying to deal with, what vendors might be useful, what constraints you have to deal with. Make sense? Okay.

Well, with that, we could then answer questions that we've been wanting to answer for a long time. Questions like, how secure are we? How secure should we be? And for the, how secure we should be.

We, we can look at those recipes. We can also look at those threats. And then lastly, how do we get there, where we look at the market? And we'll also look at those business constraints or allergies. Okay? All right. So with that, oh, so, okay, now we're gonna jump in shortly with the actual exercise, the workshop. But before we do that, I need to also just make sure that we understand this mapping ex. So everything seems pretty straightforward, right? Yes. Anyone confused any questions yet? Okay.

'cause you'll have a lot of questions after when, once we start to exercise, because where things map is not so simple, it's not so straightforward. Okay? And because I know that we're dealing with, again, I'm, I'm American, so I only speak one language. American.

There's, we're gonna have a language challenge. We, we may have a language challenge here, just because I don't speak German, but let me also just say we have this language challenge even amongst those in the US that all speak in English, okay? Because we have these words that we use, but we don't actually know, we don't have a common definition for what these words mean, let alone across different cultures, different environments and whatnot.

And so I'm gonna, it's almost, you don't have to even worry about the words, just call 'em function one, function two, function three, function four, function five. Okay? Because we get thrown off by what the words say, and we think it means something else. Okay? So for example, the word identify versus the word detect, okay? And the English language at least, they're relatively synonymous, and we interchange them all the time.

In fact, nist, which is one of the agencies in the un United States that defines standards in their definition of identify, they use the word detect in the definition of the word detect. They used the word identify. That is not very helpful, okay? Because it creates, it creates this confusion around what exactly these things are.

Well, one of the ways that I try to remove this confusion is by having a very clear distinction between left and right of boom. So boom occurs between protect and dtif. Everything before that is under the functions of identify and protect. Everything after that is detect, respond, recover. Okay?

Now, let's just do a quick quiz to see whether or not you got that. All right? So let's take vulnerabilities. Okay?

Are, do we identify vulnerabilities or do we detect vulnerabilities? Okay. Who thinks it's identify and who thinks it's the word detect? Okay? So we have a couple people who, well, so we clearly have this agreement here.

Now, for those who raised their hand for detect according to nist, you are correct, okay? But everyone else who said identify you are actually functionally correct, because looking for vulnerabilities is something that happens before boom.

Therefore, the right word is identify, not detect. However, NIST got it wrong. So in version one of the NIST cybersecurity framework, they call it detect In version two, they fixed it, it's now identified because it's left of boom.

So, and when you talk about left of boom, we're talking about the structure of your environment. And the structure includes things like your structural weaknesses, which are, which is like a vulnerability versus situational awareness, which is the exploitation against the vulnerability.

So it's a, it's simple things like that that will help us figure out whether or not we're dealing with left or right of boom. Okay? All right.

With that, let's now dive into the workshop. So I'm gonna skip exercise one, exercise one. I left open for you all to fill in the blank, so to speak. So if there, as we go through the exercise, if there are things that you want to map that I haven't mapped for you that we haven't mapped yet, then just fill it in and we'll talk about that as a part of the workshop. Okay?

If, I don't think I have enough materials for everyone here, unfortunately, so some of y'all will need to share for those who are sitting in the back, you will need to probably have a table or something to work with. And I do have some materials for you guys to use to be able to do the exercise. So if you wanna grab some of these. And I don't know if I, I, I, I brought more books. Yeah.

Oh yeah, you can, you guys can sit here too, if, here we go. Okay. So the first exercise is mapping it. So fir the first exercise is actually exercise number two. It's sort of like your elevators here, where ground floor is not one as it should be. But anyway, nevermind. That's a joke.

Oh, actually, can hold on a second. Can you, you took two. So everyone should minimally have sticky notes and a whiteboard version of the cyber defense matrix. If you don't have a book, I have more somewhere. All right? Okay. So the exercise that we're gonna do is to map product categories.

And so I, I will, this is, this may be challenging if you're not familiar with some of these words, but I will kind of explain some of them. But the goal is to nonetheless try to figure out where they map on the matrix. So all you gotta do is take these sticky notes And put them in the box that you think is appropriate, okay?

And like, like that. All right? Okay.

And so, and you can work with your partner or some, a friend or somebody next to you if you want. But the goal is to figure out where each of these things fit.

Now, one of the goals is to try to make sure that they fit in one box. In the only one box, okay?

However, you'll find that that's a challenge, okay? And I want you to struggle through the challenge, because that's actually part of the learning. But that said, if you find that you have to put it in multiple boxes, so be it. But there's a reason why I only gave you one sticker, and that sticker's supposed to generally fit in only one box. Okay? So go ahead, try your best. For those who don't know what some of these words mean, software, SBO m is software bill of materials, XDR is extended detection and response, and all the rest, I'll let you figure out, okay?

So I'll give you about five minutes or so, and then we'll reconvene and talk about the answers, Okay? Okay. Usually detect and response though, That's Yes. But what does, what does X mean? That's part of the challenge, right? What exactly does X mean?

And, and, and for those who are, hopefully everyone here is somewhat familiar with cybersecurity. Like what?

Like, hopefully you guys are practitioners or at least have some understanding of cybersecurity. But if you're like, what are the, are these words, I didn't make up any of these. These are all like, market terms.

Oh, hey. Oh, I should mention don't, don't use the answers in the back.

Hey, hey, that's cheating. That's cheating. The answers are in the book.

Well, some of them are. Don't cheat. Forgot about that. Okay. All right. I think most of y'all are mostly done. Okay. How was that exercise? Thoughts? Just you can start yelling 'em out.

What, what were your thoughts? Easy, hard, Medium. Okay. All right.

Well, let's see how you, if you got it right. 'cause you might thought it was easy and you got 'em all wrong, so, yeah. Okay. And by the way, this is my opinion, okay? So I I could be wrong as well. All right? So I could be wrong, by all means, I could be wrong, but this is my opinion, and I'm always happy to be corrected by whoever has more and better information. So let's start with software bill of materials.

So software bill of materials helps me understand what are the ingredients in my software application, and therefore I'm enumerating all the application components, therefore, I put it under application. Identify. Did anyone have sbo m anywhere else?

Okay, data security, posture management. All right, so this one's, well, data is obvious, right? Data security, right? But it's a question of whether or not it's only identify or identify and protect. So security posture management, usually, well, most of these tools that say security posture, it's really just an identify function. It's trying to understand do you have any structural deficiencies? Not actually correct them. Okay? But some of these data security, posture management tools are starting to correct them as well.

And so that's why I am okay to put it into Protect, but most of them fallen under identify, oh, I should mention, so one of the things that I try to do in the cyber defense matrix is to have a really clear understanding of what the primary capability is. And that's really where I put, I, I map most of these things. So I should probably put data security, posture management more under ident, like shift it even more so under identify. But there's a lot of confusion that you'll get when you try to understand what these vendors do relative to their primary function versus their secondary function.

So I'll give you an example. Let's take, let's take antivirus, of course. Everyone know knows what antivirus is, right? What box do you think antivirus fits in? Okay?

So let's, let's, we'll start with the asset class. What type of asset does an antivirus protect?

Oh, address devices. And then what? Function Protect. Okay.

Now, if you read, so device protect is the most likely place to put it, right? But if you read their marketing literature, they also claim to protect your data, right? Okay. Do does antivirus protect your data? Yeah.

Well, wait, you're about to say a word. They it does.

I mean, it protects, stops, malware getting in, then stops that malware looking to dig. I, and you were about to say a word, but you, you kind of hesitated, you said, I think you were about to say the word eventually. Yeah. Yes. It eventually protects your data, right? Okay.

I, I, I, I just, I, I, I don't know if you were gonna say that word, but you just kind of almost hinted at the word eventually, but it's not, it doesn't do it directly. It doesn it indirectly. Okay? But most of these products will claim protecting your data as a secondary function because, well, it's trying to protect something else. So like a firewall protects your what function, what asset class network. But they'll also claim to protect your data too, right? But does it? And the answer is no, not directly.

And so that's where it gets, it gets confusing, and it is particularly confusing when we get to other functions. So they'll say, we'll, look for all your vulnerabilities and we'll also protect you from them, or we can help you protect yourself from those vulnerabilities. And so it's what I call in, in English, a weasel word. A weasel word is like, helps or supports or enables. They don't actually do it. They help you do it. And usually it requires something else for you to actually be able to do that function that we're, that they're claiming. Okay? All right.

So that's data security, posture management. All right. So XDR. So the DNR should be pretty straightforward, okay? Sometimes the r the response piece, that's actually questionable. Most of the times it's only just the d the the piece. But the real question is, what do we mean by X? Okay?

Now, most XDR products that you'll see in the market really only cover devices and networks. And for them, that's X. But for me, X is, I can asterisk, it's a wildcard, which means all five asset classes. But most XDR products that you'll see do not do all five asset classes. They barely just do two. But if you look in the market today, largely, XDR is in these two categories that you see under devices and, and networks, and under detect and respond. And that's primarily where, where they sit. Any questions or any, any other thoughts on that one?

Questions, objections? Have you seen anything else? How about that in terms of where XDR might fit?

Yeah, I know I cheated. I know you could tear it, you could tear the sticker in half, I guess. Don't do that.

All right, then lastly, we have identity detection and response. Now, you, this is where you could have put it, but I'll also take that answer in a lot of different places. And the reason why, and you'll later on, you'll see why I do this. But the word identity is one of those other confusing words in the, in, in our la in our practitioner, in our practice, because it implies that identity is only tied to users. And that's not true. Every one of these asset classes have an identity, and we'll talk about that a little later.

So the notion of like an identity detection response being largely user-centric is actually a mistake. But that's generally where most of those fit. Okay? And we'll talk about that again a little later. But if you put identity detection and response across multiple asset classes, I'll take that answer as well. All right? Okay. How did y'all, do? Anyone get all of 'em? Mostly right?

Okay, good. Okay.

And, and part of the idea here is hopefully as we do these exercises, we converge on the right answer as a collective group, because, well, I mean, if we're all confused about what all these things do, then you know, what hope do we have in trying to secure our cybersecurity ecosystem? Once we have a better understanding of all these things, then it helps us then make sense of what we're doing as well. Okay? All right. Okay. So the next exercise is on measurement. So in the book, you'll see that I have no, I have exactly zero mentions of the word maturity.

So maturity is another word that I, in the English language, that we abuse too much. And it's because it's a crutch for representing a lot of different things that we hope for in cybersecurity. But what I wanted to do instead was to have a much more precise understanding of what do we mean by this type of progression? And one of the ways that we can understand that is in the progression of our, what we're measuring itself, okay? And so there's a perspective that I have in terms of different types of measurements of being different quality, but also different, different difficulty as well.

We oftentimes provide a measurement that is really easy, but is also very useless as well. Okay? And we oftentimes see this manifest in, for example, security vendor questionnaires. Okay? Like for example, do you have MFA? And of course, the answer is probably yes, I have MFA, 'cause you kind of have to, right?

Okay, so here we go. Yes, I have MFA, here we go. I have MFA, right? Here's my little token. But what good is it if I just bought, I bought it and haven't really actually implemented it anywhere? And that's really kind of the perspective here. Just because I bought it, just because I have the presence of it, doesn't mean anything besides the fact that I bought it. It doesn't mean that I've actually implemented it anywhere. It's not like I've plugged it in, although I'm gonna plug it back in, so I don't lose it.

It's not like I have, I haven't turned on, I, I don't know if I've turned on all the features. Do I have origin binding turned on? Have I turned on all the necessary features to make it phishing resistant? Then how well does it work? And oftentimes, that's the kind of question that we really want to answer. But if you don't have some of these preceding measurements, then how good is your actual calculation of performance?

Lastly, how efficient or what's, how cost effective is the capability? Again, those are questions that we all seek to answer, but it is oftentimes difficult to make a claim that we have those kinds of measurements if we don't have all the preceding ones as well. And to make it very clear as to why that's the case, consider we have two situations where I can say I have on the, okay, so in the red, on the far right, I have something that's 60%, I, I make a claim that it's 60% efficient. And what I'm trying to accomplish, that sounds great, but you realize my coverage is pretty weak.

So my overall efficiency is only 12%. Okay? On the right, my efficiency is 30%, but my coverage is pretty good across most of the things.

So, so, so my overall efficiency is actually 27%. We can use these numbers to lie all day, right? And that's what we do in cybersecurity often, you know, when we're presenting to the board or presenting to a lot of other folks. But when we're trying to, when we don't want to fool ourselves, we wanna have a much more systematic methodology where we have a very clear understanding of how good our program actually is. Okay? And to do that, i, I, I think it's important that for us to understand, for ourselves at least how good our measurements actually are.

Do we have all the requisite preceding values? So if we make a claim that this performs this much, this, well, well, how do we know? Do we have any other information on coverage? Do we have any other information on utilization? Do we have all this other information that we might be missing? And so that's our exercise. Okay? So the exercise, now we're in exercise three, and there are two parts of this exercise. So the first part is understanding what type of measurement it is.

Is this a, a level measurement, AB level measurement, C level measurement, or D level measurement, or AE level measurement? Okay? And then later on, we're also gonna then map it to the matrix as well. Okay?

Now, here's the thing that I want you to do. So there's a claim that quality, okay, there's a claim that's being made, meaning we, there, we are claiming that we have, let's say if I were to give you an example, let's say if I have an A level measurement, okay? If I'm claiming an A level measurement, then What you should think about is, okay, from this claim, do I have any other evidence of performance utilization or coverage?

Okay, if I have AB level measurement, do I have any claims around utilization or coverage? Okay? And so on and so on.

So if, if you read a, a particular claim, then look for all the evidence before that as well, okay? And if it's missing, just note that it's missing. The claim is still gonna be that I have a, a level measurement, but you may say, okay, I don't see any performance or utilization or coverage data to be able to support this, and maybe that's what I need to look for. Okay? So that's the goal. This one's a little bit more complicated, but hopefully you'll get a sense of how this works. I'll give you guys about maybe 10 minutes to go to this one.

If you walked in recently and don't have materials, I have materials for you here. All right? So 10 minutes-ish, and we will reconvene. By the way, this is a workshop for those who came in recently, which means that you have to come, you have to actually work. Which one? Yes. Thanks. Welcome. I think I'm almost out. You guys can share? Yeah. What's that? Is This exercise Three? Yes. We're in exercise three. Still use this Matrix. Not yet.

So, oh, yeah, I'm sorry. Thank you for the clarification. So initially, before, yeah, before you see where the boxes are, get a pin and write the letter, the associated letter in the box, what kind of, what kind of measurement is it? So let's say for example, we'll walk, we'll walk through one of these together. Okay? So we have D, l, P, okay? Because I'm sure you've run into a, you may, anyone who's filled out a vendor questionnaire, that's a question, right? Do you have DLP? Anyone see that question? And of course, the answer is yes, we do. What type of measurement is that? We have DLPY.

You can yell it out. E it's a presence. Measurement doesn't say anything else about whether or not you actually have turned it on anyone's monitoring logs. You've turned on the features. That's all it is.

We have, we bought D, LP. Okay, so what you would write in the, in the sticky note is E, okay, in the little box, write the letter E, so then look at the other ones and figure out what the associated quality of the measurement is. Then later on, we're gonna, oh, actually not later on, you can, you can do this as well, which is take that sticky note and stick it in the right box as well. Okay?

Oh, All right. Can't find one. But So who here missed the intro? Who here missed the intro?

No, no. I mean, the intro, intro. I know you guys, I know many people have missed it because I, I'm, but you're not raising your hand, so, oh, sorry. The very, The very beginning intro. Okay. All right. I know who you're, who's lying because I, I, anyway, okay. For those who missed the intro, I'm gonna for the, if, if you want saw the intro, keep wa keep working. But for the sake of those who missed the intro, let me go back and just share the intro again. Okay.

So again, this is just for those who missed the beginning. One of the challenges that we have in cybersecurity is that we have a lot of, it's very difficult to parse through all the buzzwords that we see in the cybersecurity ecosystem. And so what I did was I came up with a simple structure. The structure consists of these two dimensions, things that I care about, things that I do. I organize this into this cyber defense matrix. It allows me to take all those buzzwords and organize 'em into something that's more functionally aligned.

And so once I can align them functionally, it allows me to see those gaps much more easily. Before it was hard to find the gaps. Finding gaps here is really hard. Finding gaps here is much easier. And of course, I can map vendors and those vendors help me understand what they do as well. There's a lot of different use cases that this cyber Defense Matrix has, and there's tons of historical briefings that I have on all this. But more interestingly, the cyber defense matrix is a nice system for organizing information. And as an example, I organize different types of categories of information.

So for example, what are your current set of capabilities? You can think of that as what's in your cupboard? What are the practices or what are the requirements that you have to meet? Think of those as recipes. What threats do we have to deal with? Think of those as nutritional needs. What's available in the, what kind of, what, what do the vendors outside do? Think of those, those that's a supermarket. And then lastly, what sort of business constraints do I have to deal with? Think of those as allergies. Each of these layers can be mapped to the cyber defense matrix.

And so, as an example, here's a mapping of all the controls that we see in the CIS critical security controls. These are like recipes. There's different types of threats like phishing, ransomware, and so on and so forth. Here's a representation of all the controls that you need for ransomware. Now I can start layering different things. Like for example, here's, here are controls that we need for ransomware. Here are the potential vendors that can help. I can now organize these threats based on severity and figure out which ones I need to prioritize.

I also have to deal with, I also have to deal with allergies. So different business units have different allergies, different constraints that prevent me from being able to implement all the controls I want to be able to do. So if you have, and, and each different business unit has different allergies, so you gotta just be mindful of which ones they are. But you take all those and then you can layer them on to say, here are now, here's a com, a composite view of my threats, my controls, and my, my threats, my control requirements, my business constraints, the vendors that I have.

And so with this, with this composite view, I can now understand questions like, how secure are we today? How secure do we need to be? And how do we get there? So that's what the cyber defense matrix is all about. And we are now in exercise three. Okay. Questions? Alright. Okay. We have about three more minutes of my 10 minutes, but it looks like most of y'all all are done. So I think yes, anyone not done yet?

Alright, so let's walk through the first part of the exercise in terms of the quality of the measurement. Actually no, we'll do the, we'll do the second part too. Okay. If you haven't already.

So has, has everyone mapped their, mapped them as well? Okay. Not everyone has mapped them, but we'll go ahead and go through their exercise. We'll go ahead talk through the answers there as well.

Alright, so As I mentioned the example, so this is what you would do at the end. You'd have, like if we had a example like we have MFA, again, that's an E level measurement that just tells you you have presence, doesn't tell you much more else than that. And then MFA maps to user identify. I'll talk about that a little later as to why that's the case. But here are the answers that I had.

Again, this, this, this is my opinion. I could be wrong. You guys can have a different opinion on things. When I marked it with the asterisk, it's where I would say it's AB type of measurement, but it's, I'm lacking the additional qual information to make, to have a really strong claim around whether or not it's actually correct. So let's walk through some of these. We'll start with, we've scanned 98% of our code base for vulnerabilities.

So again, looking for vulnerabilities is an identify function. It's talking about code. So that's applications. Now it's saying we've scanned 98% of our code base. That's a coverage measurement. So hopefully you have D for that. And that's under application. Identify. Anyone have something else for that? I'm sure somebody has something else for it.

Okay, what did you have? And, and you wanna make an argument for why you thought a bit different? Chose performance?

Oh, you thought it was a performance measurement? Yeah. 'cause like we have covered 98%, which is like kind of, you know, performance as well.

Or maybe, I don't know. What if you found no vulnerabilities, Blue?

Well, if you, if you found, if you scan 98% of your coding, you found no vulnerabilities, would you think that your vulnerability scanner was actually working? Probably not, right? Just because you know, there's always vulnerabilities in, in your code. So what I would say is, and, and here's the thing that I would say, if, if you're looking for a higher level measurement, then let's talk about what that look measurement actually might be, right?

So for this one, I would want to look for, if I'm trying to move the needle up in terms of a, a better type of measurement, it would be, is it looking for, let, let's, let's use some examples around like oasp top 10, whatever, whatever, okay, does it have coverage? Does it have, does it, can it cover all the different types of vulnerabilities?

Okay, what about code ba? Like what about different languages? Does it cover Java? Does it cover TypeScript? Does it cover all the different types of languages? Those are kind of questions that are left unspoken here for which you would wanna have clear perspectives on, because otherwise, again, there there's not much to it. It's a claim that we're someone's making. And by the way, oh, I should say these are all, these are not made up claims.

I'm sorry, these are not made up metrics. I've seen these in various security reporting. I've seen this, some of these in board reporting, which is kind of dumb. But nonetheless, there's, there's, I've, these are not, I I did not pull these out of thin air. These are numbers that I've seen people report where I said, okay, let's, let's see what we can do to do better here.

All right, next one. Our phishing click rate is 25%. So first of all, let's see, how did anyone map this somewhere else?

Okay, where? DD Oh, I'm sorry.

Well, okay. Actually no, sorry, not mapping it, but the, not the grade, but the mapping, like where it's, where it fits on the box. Did anyone put it in different box? What I put it on detect.

I mean, it didn't say anything about, okay, is this, is this mock fishing? Is this real fishing?

Oh, yes, yes. A very good point. Very good point. So if it was mock phishing, Then, then it's, then it's Identified, then it's identified. If it's real phishing, then it's detect. Now what's the difference?

I, I mean, kind of what like, so if it's mock phishing, you should think of that as a vulnerability scan of a user. Okay, how do we vulnerability scanner, we, we vulnerability scan devices, we vulnerability scan applications, and so on and so on. Think about what do we use to continuously vulnerability scan our users to see whether or not they're susceptible to phishing emails. And those are mock phishing emails.

So to see a clicking a, a phishing click rate of 25% of our vulnerability scans against our user is a representation of our users' vulnerabilities or user susceptibility to vulnerabilities. I met mock, so that's why I put it there. But you're absolutely right. If it were our real click efficient click rate, then that would be perhaps more on the detect side. You could Argue though, that's a fake vulnerability.

Well, it's still a test. Like, so if, if it's a te it's a simulation against it, it's a scan, right? It's another form of a scan to see whether or not the a a entity is susceptible to a vulnerability, by the way. So how do you patch a human Training training?

No, you don't fire them. Well, if, if they don't take, if you don't patch a device, if someone, if, if you find a only building a device and they don't patch it, what do you do with the device eventually or isolate you and eventually you say, we've gotta get rid of this thing, right?

So I mean, you could take a similar sort of view with outside of the workers' council getting in the way. I'm sure you can do something like that, right? Okay.

Anyway, as far as the grade is concerned, someone said D right? Yeah. Okay. This Is, this is not performance just tells you that. So if you want to like, relate performance to the, to the level of the level of awareness of your employees, I, I, I, I saw that in the context.

So I, I think this is pretty, pretty, Yeah. So I agree. I agree. It's useless, which is why I put it in here as an example.

What are, what are ways to make it better that actually el So I put an asterisk there to say, even though, well, again, I can, we can debate on the specifics of where it might be in the ranking, but we, we hope for better measurement, right? So what would make this a better measurement to really truly be AB? Like what kind of, what additional information can you measure? You have to put it in a timeline, right? Okay. Yeah. You have to, you have to, you have to measure like 25% is too much number down.

And continuous testing actually, look, if your numbers increase, Actually, okay, so still, So if there is just one person left context of risk. So let me zero. Yeah.

So let me, let me make an argument that maybe this number is actually really good. Okay, Let me, let me rephrase it. Okay.

Our, our mock phishing click rate is 25% for phishing emails that simulate a nation city attacker with highly targeted phishing content. Okay? Okay. And if I had that as a precondition 25%, I'd be like, wow, we're doing really well. Okay. Yeah. Okay. Okay.

But my, my point is, I, I've caveated it to say, let me, it's not just that we've done a random phishing test, it's, we've done a very highly targeted type of phishing test, which then get into more around the qu the type of quality measurement that we would see for that would address like the, like AC quality measurement or d quality measurement. So what are, if I'm, for example, in other words, another aspect I'm saying is our phishing click rate for the, of all our admins, okay? Versus our phishing click rate for our entire employee population. Very different, right?

Or our phishing click rate for our highest risk users. Again, all those are very different types of measurements, which give you lots of different value in different ways. This is not very valuable because it doesn't have any information on coverage. It doesn't have any information on the sophistication of the phishing emails. And those two additional date details could make this measurement way better. It'd be interesting if you could measure somehow what people do. They just ignore them, something else would happen.

But that would be a measure of awareness, whether your security awareness is working because Yeah, so the Tell your security, Yeah, the question is, or the statement is, do we, if we have actual data on how well our users actually respond to real phishing emails or simulated ones, that's actually additional detail that's helpful in understanding the strength of our security program. Or let me rephrase it slightly. It's additional detail that helps us understand how well our users are protected.

So everything that you just mentioned then falls into measurements associated with User protect. Okay? What's the actual reporting rate?

Are they, you know, are, are, is, is security? So if security training and awareness falls under user protect, what are the measurements to say that that program is actually working? Okay. And some of the examples that you gave fall into user protect. Okay. All right.

Questions, thoughts? It's making sense. Anyone confused? All right.

Okay, let's keep going. So I think we already talked about you, we have DOP is a pretty crappy measurement now in terms of where it fits, what, what do you think? Did anyone put it somewhere else besides data protect?

Okay, where'd you put it? Data recover. Yeah. Okay.

And why, why would it be data recover? DLP. It's a data loss protection. So it should be recovered Or what?

Wait, just call it data loss. Yeah. Protection. So why wouldn't it be protected?

Yeah, but it's the process that it should be taken care. So the data is being lost and it is been protected, so we need to respond it and recover it. The data. Alright. All right. Okay. Who thinks, let me, let me generalize it a little bit. Who thinks DLP is a ride of boom tool? Anyone? Okay. Who uses DLP as a ride of boom tool as a right of boom meaning?

Okay, Okay. Right. So for those who missed my earlier conversation about the left right of boom, so right of boom, meaning it's used for detect, respond, recover. And so part of what I think you're making a claim is it's a right of boom tool, specifically recover. I would question that, but most of us do not put DLP in blocking mode. Why? Because we will get fired.

Okay, but does that make it a detect tool? Let me ask you, we have firewalls, right? And we have firewalls and we can turn it on blocking mode and we do for, you know, very obvious bad stuff. But for many other things we may actually put it in just monitor mode, right? Does that make a firewall, does that move a firewall from network protect to network detect?

Okay, so now let's take DLP again, DLP is a protection tool. It's data protect that's, you know, data loss protection. Just because we turn it into monitor mode and not block, does it all of a sudden become a detect tool?

Well, I'd argue it doesn't because what do we do? We take the log, we put the log into the seam or something like that, and the seam is the detect tool. Awesome. That per exactly right.

So again, the, the DLPA firewall, av all these protect tools log and sometimes block, but across the board they log the logs go into an actual detect tool like a siem. And by the way, I, I didn't mention this, but this is actually pretty important. The thing on the bottom, the dete degree of dependency on people process and technology.

I, I don't know why I don't mention this more, but that's actually the, probably the most interesting part of the matrix because we get this wrong all the time. We think we put in a SIM and the SIEM works perfectly fine without any user intervention, right? And we quickly realize no, it needs constant daily feeding and tuning and adjusting, okay?

And it's, it's, that's done by people. All right? And so we forget to resource detect tools appropriately, making sure that there's an equal amount of people time as much as the, the cost of the technology as well. But anyway, exactly right in terms of the function of the ADLP tool is not a detection tool, it's a logging tool and the logs feed into a detection tool. Okay? But because, but just because we put it into monitor mode doesn't all of a sudden make it a detection tool. It doesn't make it fall into that detection category.

And we, we, again, we confuse this all the time. And the reason why we, the reason why this is important, and the reason why it's good to not re forget that is because if you buy a, if you buy DLP because it has detection abilities or you think it does and you don't actually buy a sim or you don't buy an a, an aggregation tool of some sort, some sort, then you are in for a world of pain, okay?

It's, it's, you're not gonna actually be able to effectively perform your detect function just because you're gonna be overwhelmed, you're not gonna have any correlated logs, you're not gonna have all these other sort of things. And so that's, that's where we again, think, okay, just because I bought this and because they claim detection, all of a sudden you think you have detection when you don't actually have it. Okay.

Alright, let's go ahead, go to the next one. Our firewalls have blocked 45,000 attempted connections. Okay? I saw this as a performance measurement as well.

But again, a a very weak one, okay. A very weak one.

In other, in other words, it's saying, Hey, we've, we're, this thing is working because it's blocking stuff. Okay, that's it, it's all about it. That's about it, right? So the question is, what can make this better? What can make it a true, like what, what are the coverage and utilization measurements that would actually make it better? Some analysis? Yes. Sure. Yeah.

Well, well, I mean, so it's kind of, I'm, I'm, I'm trying to not make fun of too many people, but I, I've seen, I've seen measurements like we've blocked a hundred percent of all our attacks. Like one, you've blocked a hundred percent of your, all your attacks.

Like, okay, I'm not sure what the denominator is, and I'm not sure what you classify as an attack, but I've seen those kind of claims and I'm, I'm sure you've seen, we've been attacked 45,000 times, okay. Or 45 million times or whatever.

You know, these are like stupid claims that we had to come up with in the, in the day, in the past days. I hope we don't make those claims anymore. But these are real claims that we've, I'm sure some of y'all have seen the, the more challenge for us is one, let's not try to use these crappy measurements anymore, but two, well, how do we make these better so that they're actually useful to us as practitioners?

Okay, I'm gonna go ahead and keep moving on the, we've had seven SLA misses for patching critical application vulnerabilities last month. So I put this under application protect and I, I actually liked this measurement because it provided the necessary conditions of what I'm actually measuring. Okay? There's a timeframe, there's a scope around which applications I'm trying to patch.

There's a, there's a defined, presumably there's a defined SLA, it doesn't say which SLA misses, like it could have been, oh, actually, I guess it's for critical, critical SLAs anyway, but I, I, I like this particular measurement 'cause it had the necessary pieces to provide a perspective of coverage, meaning critical apps. And I, I'm not sure what the utilization measurement is, but you can see, hopefully see why it's a, it's a better sounding measurement than all the other ones that we've seen so far. Okay. Did anyone have any questions about this one or thoughts to make it better? Okay.

All right. Last one. Our main TAM containment for is five minutes for intrusions in our DMZ. This is a network respond.

And again, a b-level measurement, a performance measurement. 'cause it's saying how good we are. Containment measures are the, it's scoped to the DMZ. Yeah. Okay. Anyway. Alright. Questions on the measurements piece? Alright, so my, my challenge to you guys is take your measurements and try to map them to the matrix. See how you do. Some of them are, well, you'll start questioning your metrics and your measurements after you go to this exercise. All right? Okay. Last one is mapping zero trust.

So a big buzzword as we all know, but I wanted to give some framing of how I think about zero trust because it, it, it provides I think, more systematic view than I think most have will have seen. And so I mentioned in the upfront portion, I, I shared this cyber defense matrix with everyone back in 2016. And I talked about different shortfalls where I couldn't map orchestration, analytics, and governance. So you take these five asset classes and take these three foundational layers that I couldn't figure out how to fit. And you organize it this way and you have Thea zero trust maturity model.

So if you are familiar with the zero trust maturity model that the US government has put for forth, the foundation is actually the cyber defense matrix. Okay? So anyone familiar with the zero trust maturity model? If you have, if you're not familiar with it, definitely just search on it. You'll find really, it's a great resource. But the thing I wanted to point out that you see from the change from here to here is that users becomes identity, okay? But the icon is still a user, okay?

So it implies, again, that only users have identity, which is a problem because all these assets have identity, not just users. And because of that, we have to think of, we have to think about what is the identity of all these things as well. What's a device identity? What's a network identity, what's an application identity and so on. And when we think about, when we think about that, it'll help us then map things appropriately in the matrix as well. Okay? And that's actually the next exercise.

So I'm gonna, I'm gonna let you guys struggle through this and then when we talk about the answer, you'll see how everything fits together. But the struggle is the part is the good part. So four categories that are in the zero trust area. See what you can do to try to map these four into the cyber defense matrix. And I'll give you like four minutes here for those. Anyone not familiar with passkey? Okay? So I'm not gonna try to give you an answer, I'll try to tell you in a way that doesn't give you the way the answer.

But passkey is a passwordless mechanism that has been introduced recently and more, there's been more mainstream adoption, but it's passwordless authentication where it stores an authentication credential on, Hmm, it's, it stores authentication credentials on the things that you use so that you don't have to use passwords. Okay? All right.

Okay, let's wrap it up in about 20 more, 20 more seconds. All right. I'm not gonna show you the answers right away because I, I'm just curious what people actually, where put people put things. So first two FA token, where did you put two A token users and what asset class?

I'm sorry, what? Function. Protect.

Identify, okay, between both. Okay. Did anyone else put besides users, did they put it anywhere else? I Put it on the devices. Devices, okay. Others Applications.

So, so we think that two FA tokens identifies users and protects devices and applications and, and the user themselves, right? That's where we think it fits.

Okay, I can, if I can only pick one box, which box will that be? Of all those things. User and what?

Identify, so remember the whole idea about primary versus secondary. Okay. Like what's the, what's the first thing that the, that the two FA token does versus what does it, what can it do after that? Okay. So this is a situation where we confuse the primary versus the secondary capabilities. The primary capability of A two FA token is to identify a user. The secondary capability is to be used as a mechanism for protecting all these other types of assets. I'm not exactly sure I'm agreeing with you on that.

'cause all this thing does is it says I am a device and in the background someone says, okay, this as part of the identification for this user, he must show this device. Or he must let this device interact with the log on mechanism. But it still doesn't say this is alpha, it just says I'm a device that is supposed to be owned by.

Okay, so let me ask you another way to think about this is you kind of hinted at it in the way that you described it, but there requires something else to be able to perform the protection function. Okay? And if something that, if in your mapping exercises, as you go through the mapping exercise, if you see that there's something else that's required, if there's, if there's you, you think it's gonna protect your application, but it requires something else to actually protect that application, then again that's a secondary capability for whatever you're trying to map.

So a two, a token by itself does not protect your application. A two a token with something else protects your application. But that something else is actually what protects your application. Okay? So let's actually talk about that with API gateway. So API gateway is something that actually protects your application and it may consume a two a token to do that. So if you're trying to actually protect your application, that might be that. Now API gateway, you don't necessarily want to use a two-factor authentication with API gateways 'cause it'll break.

But the perspective is there is a a, if you're trying to protect a device or an application or a network or data, then that's something that requires something else. And you usually something like a gateway or an access proxy. So let's look at zero Trust Network Access Proxy network, right? Hopefully. And hopefully got also put that into Protect as well. Did anyone put it elsewhere besides Network Protect? Hmm? Devices as well.

Oh, devices as well. Okay.

Alright, so this is where, so this is, okay, hold that thought and I'll talk about that in a little moment because you are correct in how many people think about it. Also, zero trust, network access. People think of that also as protecting your applications as well. Okay? But you'll see a little later where I break that out into something more specific. And then lastly, passkey. So passkey is a little confusing, but anyone, where do people put passkey? User identify. Okay? And I generally will maybe accept that except that passkey as designed are supposed to be tied to your device.

Now, things like iCloud will abstract that and put it on all your Apple devices, but it's actually tied to the device. So in my view, again, my opinion, I could be wrong, is pasky is device. Identify the two. A token is user identify. And then you see where the API gateway and the zero trust network access proxy is. Okay.

Now you, you brought up a good point around zero trust network access and it being a, something that provides access to other things besides the network. And that is the category of zero trust. Access covers that. But we are oftentimes imprecise in our understanding of what zero trust network access is. Okay? And to be able to understand that there, I thought through this in a more systematic way. So we look at the functions of identifying, protecting, you can think of one side as a from and the other side as a two. On the left side, it's the authentication.

The right side is what makes the, what does the authorization. So on the left side, think about all the things that we can use to identify who you are or what, what it is. Okay?

So again, all these asset classes have identity. And so in the current and how most people think about the current set of market offerings, we typically, this is what we typically do.

We, we have some sort of device cert or passkey. If you're, if you wanna authenticate the browser or the application, you can use a secure enterprise browser, a way to authenticate your i your network as like IP address and of course username and password for the user themselves.

Now, in terms of what IT access is, okay, there's all these different access proxies that we have. Virtual desktops for applications instead of zero trust, network access. What I see is zero trust application access, okay? For networks, of course, zero trust, network access. And for data, we still also see something similar.

It's, it's, it comes, goes by different names, but you have data access proxies and, and so on and so forth. But the perspective here is that this is just a partial way to think about, or this is what we see in the market today is a partial way to think about what's available from an identity standpoint. There's lots of different types of identities we can put forth, okay? And we can go, we can go overboard here too and add tons of friction. But if it's a machine you theory, you can actually ask for a lot of these things and not really care because it's a machine.

But it takes more overhead, more orchestration efforts to, to make sure that these systems are communicating with each other in such a way that you know exactly who and what it is. Now the way to think about that is also each of these have varying levels of trustworthiness. And I can go from a scale, this is just an arbitrary perspective, but from, if I had three different levels where zero is I, I'll take anything to two being a much more rigorous form of authentication, I can set different levels of trustworthiness saying, okay, I want more of these attributes on the left.

But depe, it depends on what you're trying to access. So for example, if I'm just trying to log in and check my balance, maybe all I need is very, a few level, a few identity attributes. But if I'm accessing APII database, then I need a lot more attributes on the left side to grant you access. Okay? And the perspective here is that on the left side and the right side, we have essentially a mesh. Okay? From device A to accessing any other resource, I can, I can have any number of combinations of things where I provide some form of authentication to an application, let's say.

So if I'm trying to access application H, I can use a form of identity from my device, a form of identity from my network, a form of identity from my application, a form the application. I'm using a form of identity from the data. I'm sending a form of identity of the user themselves. And the perspective is all these different forms of identity can be used depending upon your level of paranoia that you have around whatever resource that you're trying to restrict access to. All right? All that's kind of all that might be a little bit confusing, but I can go into more detail if folks want.

At the end of the day, you have a lot of different zero trust capabilities that make up the market. First is on the left you have identity. So everything on the left is about identity. Identity of the device, identity of the application, identity of the network, identity of the data, identity of the users. And then everything to, to the right of that are the access proxies that actually do the protection function. And this is where I mentioned zero trust network access is only one part of that, okay? Right here. ZTNA.

If you think of that in a very strict standpoint, that is for zero trust access to the network, not to anything else. If you want to give zero trust access to something else, like your application, a business application, then instead of A-Z-T-N-A solution, you should actually be looking for A-Z-T-A-A solution. Okay? If you want to grant access to a specific server, you don't really care about what network it's on, you don't really care about what application it's running, then you would have something like a zero trust device access and so on, so forth.

What we would love to have as practitioners is one thing that consolidates all these different types of zero trust solutions. I don't know if that's actually feasible, but I think that's kind of where SSIS is heading.

I, I don't know if it's really gonna happen, but that's how I would map something like that. All right. Any questions? Sure. Excuse More about governance. Challenges of governance. It's interesting, you have different practitioners waiting through all the various differently, like the mapping you have your somebody to do comparable, say, Hey, we're doing this over here. This has this level of integrity, and come, this industry standard over here of care is a little bit different. How do we evaluate whether that's sufficient for us to engage with that other party? Feels like it's going?

Governance, governance. You have some comment that in terms of using this approach for multiple practitioners, an organization in different sector, try to figure out if the thing is comparable or acceptable to based on their own Or based on their risk tolerance for that organization, right?

And, and we all have different, varying, I mean, and this is kinda represented here, where again, logging into a bank account, we actually are in, in the past we didn't care that much about authentication because if all you could do, all you wanted to know was your balance, then we would have a very low bar for that. If you wanna start moving money, we have a higher bar. If you wanted to move a billion dollars, then we'd have a much, much higher bar.

It turned out though, that we had a, again, we had a really low bar for account login just to check your balance and attackers abuse that to then do other interesting things. And so we had to actually raise the bar the, to your, what, what you're pointing out is what we thought was a sufficient bar of trustworthiness that was needed in the past because of new emerging things that we hadn't considered. Now we have to raise that bar. And that was kind of done initially company by company, but eventually became more of an industry-wide sort of standard.

I think that's what you're trying to get at. I don't know if we know what that every company will start off with their own sort of risk tolerance around cost versus benefit and so on and so on. And I should say, oh, this is just conceptual. But the cost to implement is, it's sort of like the Richter scale.

It, it's 10 times harder as you keep going up higher levels of trustworthiness. So it, it takes effort, right?

To, to demand this additional trustworthiness from your, all these other types of assets. So anyway, I, I don't know if I fully answered your question, but that's, I Think you did, and you may, is that I was kind of alluding to does this give vocabulary talk to other people about what they're doing versus Oh, yeah.

Yeah, it does. It does. Yes. Yes.

And, and, and in fact, in fact this, this sort of, this specific diagram was shared at, at least amongst financial institutions to be able to say, you know, what's, what's a, what's an what, what should we as an industry kind of calibrate towards, right? And so to your question, to your point about does this become an a way for us to all calibrate our own expectations and, and measurements or, or calibration of trustworthiness, yes, it was 'cause we will hear, oh, you know what, we should, we should move this from zero to one. Okay? And we would debate as to why and the cost.

And eventually we, we realized, oh, that's a lot easier than we thought. Let's go ahead and move that in a notch up or whatever. It gives You a map move above practice. Sure.

Okay, so now let me ask a question here. I wanna show you this right here. Then I'm gonna go to another slide slide just to show you as well. So gimme an example of where we need. So data should have identity and it's presented to the user, okay? Where data should have identity and it's presented to the user as another way to think about it here, it's this, it's this representation here. Oops. So data to a user. So does anyone have an example of a failure of that? Where that's where that's failing for us today?

Where we would, we wish we would have some form of data identity when it's presented to a user? What kind, what kind of certification Data, form of data. Sure. So gimme an example. Do we know can, is there Like an employer's looking for college degree?

I was, Okay, All right. Is there a, a very prevalent form that's happening nowadays? Fake news, deep fakes, disinformation, all these things that we're seeing where we have unauthenticated forms of data being presented to a person that doesn't have good access controls for that data to determine whether or not that data is authentic is exactly that problem. Okay?

So I I, I share that just to say this is a flexible model that helps you think about lots of interesting problems that even go beyond. You can argue it's, it's still part of cybersecurity, but it, you can see how that's something that works for something even like DeepFakes.

Alright, so just to wrap up, again, I, I wanna reemphasize George Box's quote that all models are wrong. And again, the cyber defense matrix is definitely one of those. But I wanna also make sure that you understand this is a strategic framework, okay? And when we think about strategic frameworks, I also want you to think about this from the standpoint of how we look at the world, okay? In the physical world, we have this nice continuous mapping from something that's very local to all something that's all the way at the global level. And we can connect them all together, okay?

I don't know if we can fully connect them in cybersecurity, but at cybersecurity we have similar levels as well. The cyber defense matrix is a model that works at the 50,000 foot level. And I hope that eventually we'll be able to connect it all the way down to the 10,000 foot, 1000 foot level. But we will struggle in representing it properly. As you get further and further down, the information that you see at the 1000 foot level will get very, very crowded and very messy at the 50,000 foot level.

Imagine if you pulled out an atlas or even Google Maps and you zoom out, but you see just roads everywhere. You, you, it, nothing will be recognizable. It'd be completely useless, okay? And so that's the danger that we run into with the cyber defense matrix. If you try to jam too many things into it, it will get very crowded and won't give you the type of strategic insight that you need.

And it is, it is very similar in terms of even how we think about common words like risk. Okay? The word risk means different things at different altitudes. We use the same word. But what you mean that by risk at a 1000 foot level is very different than what you mean at a five 50,000 foot level. And certainly very different than even what the board of directors would expect. And the board of directors are at the, you know, a hundred thousand, 200,000 foot level.

But at the very least, we have a way to be able to, we wanna know what altitude we're communicating with and we have a way to connect the dots from above and below. But if you try to jump two levels down, three levels down, you'll start running into issues. Okay? So just be aware of that.

Again, all models are wrong, but some are useful. I hope that through today's workshop you see where it's useful, but you also see where it's wrong. But that's said, if you find other use cases, please let me know. Please share. I would very much love to hear feedback on where you find it useful. I wrote the book because I wanted to share these use cases, but more importantly, I wanted you to share use cases with the community as well. This is how to reach me, and thank you very much. If you have any questions, I'll be around.