Analyst Chat #143: Vulnerability Management: Emergency Patching and How to Deal with "Zero Days"

Welcome to the KuppingerCole Analyst Chat. I'm your host. My name is Matthias Reinwarth, I'm the director of the practice Identity and Access Management here at KuppingerCole. My guest today is Christopher Schütze. He is my colleague practice director in the area of cybersecurity and the CISO here at KuppingerCole. Hi, Christopher. Good to see you.

Hi, Matthias. Good to see you again.

It's been a while that you've been my guest here, but that shows how cybersecurity can keep you busy and keeps you from attending podcasts, for example. But this time this is of importance. We are in Cybersecurity Awareness Month, in anticipation of the upcoming Cybersecurity Leadership Summit in Berlin in early November. And this is one of the four episodes that we planned for Cybersecurity Awareness Month. If we think back, we have been doing an episode a year ago or so about the topic of vulnerability management and this will be an important topic at CSLS as well. So we want to drill a bit deeper into that and I want to talk to you and I want to have your expertize on the topic of zero days and emergency patching. First of all, what is this and why do we need it?

Yeah, this is really an essential question. When you start to talk about vulnerability management and good that you referred here to our previous episode, even if it's one year old, it's still very actual and up to date knowledge about that. You will have always the challenge that you have to patch systems within a very short timeframe. So you have to ensure that you can install something like an emergency patch, what you you mentioned something like in security update from an important server, vendor, something like an update for your core database, something like that. This is a typical emergency patch and here you have a lot of challenges. We will talk about it, but also the famous zero day. So something investigated by hackers, by professionals, by IT security professionals that really look into systems and try to detect vulnerabilities before they are somewhere else, somewhere used. Also the bad ones sell them in the in the Internet or in the darknet as potential entry points into organizations and all vendors are very used to really fix such zero days as fast as possible. This is not always a technical solution, this could also be a solution in configuration changes, and this is something you need to be prepared from a process perspective. If you are really big organization, have a lot of servers, have a lot of applications and you need them to run. It's not that easy to turn off some database which is for real time transactions. You cannot update it during the day. And if you work globally all over the world, that's a problem. And this is something you need to prepare yourself. And here, especially emergency patching and zero days are really relevant topic for your vulnerability management process.

And you said the word vulnerability management and I put the focus on the management. That means that you need to have processes in place and well-prepared processes in place before things happen. So what are the challenges that these processes need to deal with to be prepared for an upcoming zero day or any other good reason for having an emergency patch to be prepared?

Yeah. The important question here is, do you have something like an incident management in place? Do you have something like a security operation center in place? Where do you receive the information for such zero day, such emergency patches? So typically, zero days are delivered by some specific feeds. There are specific websites that inform you you can buy a service from enterprise organizations that deliver such zero day information. So this is one part where you receive a potential emergency patch. But on the other hand, you also have your normal every day vulnerability management. So the stuff we discussed one year ago. So you have scanning tools, scanning your IT environment, scanning your IT assets and detecting whether there is something needed to be changed in the configuration or some hotfix, some patch needs to be installed. And for both of them, you need something like an first entry point. Sometimes manual, sometimes with some kind of basic intelligent algorithm which detects whether it could be an emergency patch or not. Because, going back to the normal vulnerability management process, if something is defined as a normal vulnerability patch, it could be for your organization. It could be highly critical and really relevant. And you need to treat it as an emergency patch. And so it must be integrated, or handed over, that's a better phrase, into your emergency patching process. So same point in time as you deliver or receive an information from such an commercial or official zero day feed. And that is the entry point you need to define a process. Which sources of potential emergency patches do I have and how do I investigate them first? Is it based on a metric? So usually in normal vulnerability management you have a classification, the CVSS score could be a foundation that you use as a metric to identify, and you treat every day as highly critical, for instance. But nevertheless, you need experts who have to look on these things. Is this really a relevant zero day? I mean, it could also be that you receive a zero day for your internal test environment, which does not have any kind of public Internet access, only internal test purpose, no relevant critical data. And then it's not a zero day, then it's something you need to fix, you need to update, but not within an emergency timeframe. And this is something like the first step. Investigate the impact of the potential vulnerability caused by the zero day, caused by a not installed high critical vulnerability patch. And that's the first step. And I mentioned just, if you have an incident process, so the normal incident response process, something like you've been under attack or something like that, sometimes it makes sense to hand over such emergency patches, such zero day really into that process because usually, in bigger organizations this process is defined and you can hand over, you can use the existing stuff. Because the challenge is, you cannot turn off a relevant productive service, I mentioned something like a transactional database, during runtime when it is used. So you need to do a specific investigation. How to deal with that specific zero day: Can I turn off? When can I turn it off? Do I have a plan B? In the best case, you are prepared for something like that, which is another area of cyber resilience, and then you can go forward. But then you have the methodologies in place, the communication path and you are able to update your systems in a certain time frame. And the certain timeframe is really the challenge. Usually a zero day should be, in a productive system with external access, updated within 24 hours. But this is not always possible if you have patching windows once a year or twice a year and that's the challenge.

Right. So if I think, as you've just described of this as a phased approach, we have three phases. You've described the first phase. So the evaluation of the zero day, understanding that there is one, being notified that it is, applying criticality, applying to the needed measures that would be phase one, that would be preparing for the for the emergency patch if required. The second phase would be the actual emergency patch. And afterwards, maybe there are some aftermaths when it comes to cleaning up things. How do these processes work together and how do you hand over from A to B to C to get things again running? You've mentioned configuration changes, maybe also the first phase to make sure that although it is not yet patched, the risk is mitigated. What else needs to be considered while transferring from the preparation phase to the patching phase to the aftermath?

Yeah, the best thing is here to have something in place like a tool, a ticketing system, something like that, which allows you to hand over information from one stakeholder or investigative part to another one, because usually organizations handle vulnerability scanning results, not on the same level and not in the..., maybe in the same department, but not in the same level like the emergency patching or zero day stuff. This is usually more in security operations. So the active monitoring part. But nevertheless, they have to work together and hand over information, as you mentioned. For instance, we have multiple customers where we supported to implement such processes, typical ISMS tools for managing tickets with a certain level of detail where you can add additional information like, Is this a risk relevant topic? Does it need an entry in the risk registry what you mentioned, or is it solved? And after such a process is done, you can close that ticket and you have a clear statement, Yes, we implemented, we installed that emergency patch for that zero day, for that emergency vulnerability, all that stuff. But also, you can hand over it back if you investigate a potential threat as not relevant yet and can be part of the normal vulnerability management process. The stuff we discussed in the other episode.

And another really important thing is, you might have your current system environment, you have a lot of service application and all that stuff, and you identify a zero day, you do everything, hand it over to an incident management process or do it in an other kind of secure way to execute it, to implement all that stuff. But what happens if you buy another application or another server, or one server is turned off? Two months later, it’s still part of a zero day, you turn it on and the scanner identifies something, maybe this hotfix. So you need to mark specific, zero days, hotfixes, patches, whatever on a certain level of criticality. So if you turn on a disabled machine, computer server, whatever, it must be implemented or installed that patch when you turn it on again. And that is also an important part, and this is hand over from specific departments, from incident management to the security operations center, as well to the vulnerability management team at the end.

Yeah, that's interesting because that shows that it's not only this one of panic installation thing, but it's a well managed process and it takes care also of upcoming new boxes that might be still vulnerable because not being patched adequately during this panic approach, which is to be avoided, of course. So it is all part of an overall well-managed vulnerability management process integrated into incident management and beyond. So also business continuity as a bigger umbrella above that. Very interesting. You've mentioned that this will be a topic for CSLS as well. So are people that are interested in learning more about vulnerability management recommended to talk to you in Berlin?

Yeah, for sure. I will be in Berlin on-site so you can talk to me at any time. We will have a really interesting case study where we will talk exactly about these processes, these topics, how we did it for a specific customer in that case, and share our knowledge and experience here. Maybe it's also a bit easier than with PowerPoint slides, on a presentation level to share really the specific details. So really looking forward to meeting you there and don't hesitate to talk to me.

Great. Thank you. So just to sum it up, the CSLS will be taking place from the 8th to the 10th of November in Berlin and everywhere you want because it will be a virtual event as well. So it will be virtually everywhere. So you can join us and you can join Christopher in his case study on vulnerability management. If you have any questions about this topic, if you have any specific questions to Christopher, if you're watching this on YouTube, please leave a comment below this video. If you're listening to that or watching that with your favorite podcatcher in video or just audio format, just reach out to us. We are available and our contact information is easy to find in the release notes for the episode. Also on the website we are not that difficult to find. Please reach out to us and ideally meet us in person in Berlin. Thank you very much, Christopher, for your insights into zero day vulnerabilities and emergency patching as part of vulnerability management. And I am looking forward to meeting you in person in Berlin as well. Thank you again, Christopher.