Vancord CyberSound

084 - Demystifying Threat Hunting vs. Incident Response

August 22, 2023 Vancord Season 1 Episode 82
Vancord CyberSound
084 - Demystifying Threat Hunting vs. Incident Response
Show Notes Transcript

Threat Hunting and Incident Response are two tactful measures that help an organization's cybersecurity journey. But what is the difference between them?

Join us today as Jason, Matt, and Steve dive into the separate services, and discuss the proactiveness of Threat Hunting and the neutralization of breaches through Incident Response. The team also explores where forensics comes into play and how the maturity of an organization can determine what toolsets are appropriate to implement.

00:01

This is CyberSound, your simplified and fundamentals-focused source for all things cybersecurity, with your hosts, Jason Pufahl, Steven Maresca and Matt Fusaro.


Jason Pufahl  00:14

Welcome to CyberSound. I'm your host, Jason Pufahl, joined as always by Matt Fusaro and Steve Maresca. Hey, guys.


Steven Maresca  00:20

Hey.


Jason Pufahl  00:21

So I think we're, this is gonna be a, I think a reasonably straightforward discussion today. But I think it is a question that feels like, we have to answer a lot. And so we thought it'd be worth spending some time on, which is, when are you, what's the difference between threat hunting and incident response? When are you engaged in an incident, what's that distinction? Sort of, what are some of the activities that might occur for each? Sort of talking through this a bit. And I think I'll start maybe with a really, really high level definition and then turn it over to you, Matt. But I think threat hunting tends to be more proactive, and incident response, as the name implies, is, you've had an event and now you're trying to figure out how to address it, right. And all the things that addressing and recovering means, but I mean, I think at its root that's probably the simplest way to describe it, although, sometimes we find ourselves talking about threat hunting, maybe after an incident, maybe we've been called in late. And that's probably a worthwhile distinction to make at some point as well.


Matt Fusaro  00:21

Hey. Yeah, I think one way to look at it too, is that an incident response might involve a threat hunt, but not necessarily the other way around. Usually, there's a few triggers that would cause a team to want to go into a threat hunt. You know, one of them might be an actual incident has occurred, you know, that you have an attacker in your network, we know some things about this attacker, we want to see if we can go and threat hunt that to see what systems were touched, maybe we have some very specific indicators that we want to look for. Other things would be threat intelligence that comes up from whatever feed you get, maybe you want to take that intelligence and actually go search for that stuff inside of your organization. There might be an anomaly in your monitoring that happens. You're getting some alerts for things that maybe aren't, you know, very straightforward an issue, but maybe there's certain pieces, hey, people are touching this file. Let's figure out why. Let's figure out why this is an issue, why is it becoming an alert? Or you're looking for just some better knowledge about how certain things are being used. And that would be more of you have a new application coming into your environment. And you want to understand better, you know, the types of activity that's going on with that.


Jason Pufahl  02:46

So I want to pull you back to an acronym really quickly. You said indicators of compromise? Spend a second on what that is, like how do you, how do you know what an IOC might be, where are you getting that information? How do you search for it?


Matt Fusaro  03:03

There's a lot of different ways. The most common way would be a threat feed or some organization that either produces software, right, that they know already that if these IPs or file names or file hashes are seen in an environment, it's verified activity that an attacker is there. And then you can use that data to help make a decision about whether that's in your organization or not.


Steven Maresca  03:27

It's also something you develop midstream in an incident, it's unique to your environment, it's unique to the attack at hand, it's telltale signs that a system or identity has been impacted by the attack. Might not be disclosed in the way that Matt's referring to, but you know, you develop your own list that's situational. 


Jason Pufahl  03:47

Right.


Matt Fusaro  03:49

Yeah. And if you've had a compromise before, and you have some historical data to go back on, if you're a large organization or you've got assets under control that are extremely valuable, you may want to take that historical data and see if there are other, what are called TTPs. Techniques, I always forget the, 


Steven Maresca  04:07

Techniques, Tactics and Procedures.


Matt Fusaro  04:09

Yes. You want to see if you can uncover other activity that's going on from that threat actor so that you can better defend yourself in the future.


Jason Pufahl  04:17

So it stands to reason then, the better or maybe the more thorough the data that you collect and store from sort of applications and systems from a logging standpoint, the easier a threat hunt is going to be.


Matt Fusaro  04:34

Typically, you're gonna have to be a mature organization to initiate a threat hunt. If you don't have visibility into logs, into your endpoint security data, if you don't have an EDR, if you don't have a SIEM, it might be very difficult to perform a threat hunt because you just don't really have much to look at. 


Steven Maresca  04:53

I would say that the effort may be exactly the same for analysis. Whether you just stand up a threat hunting platform for purpose of incident response, or you had one previously, you still have to do the work to query, to filter, to narrow down and sort of interpret. That's probably not a great deal of difference. But you may have developed organizational acumen with respect to understanding what's normal. You might have developed filters and queries to say, alright, this noisy thing, it's not a problem, we know it's an expected behavior. So you might become more efficient by collecting data in advance. It's not a guarantee, knowing what, how things behave, you know, there's always a benefit to that.


Jason Pufahl  05:39

So, I'm going to add, I think we agreed in talking about threat hunting and incident response, but it occurs to me, where does forensics come into play? I mean, is that is that a, could that be a portion of both? Is that system analytics that might be different than data give you?


Steven Maresca  05:55

These are overlapping circles. I mean, some threat hunting platforms have forensic capabilities. Some forensic platforms have threat hunting capabilities, it's, there's an imperfect kind of definition. Bottom line, forensics is what you do when you have a really high bar for analysis, or a high effort load. So for example, if you need to determine who did what when, and there's a legal determination of a breach, or an HR action that needs to take place or a crime, you're probably in forensics territory, if you have to preserve data, contractually, you're in forensics territory, if you're simply doing analysis to figure out how things work, what files, processes, identities, things of that nature are invoked, you're probably in threat hunting territory, the intersection's imperfect, you know, exporting a forensically useful log from a threat hunting platform may help interpret some of those other types of data. So that's where they intersect, generally speaking.


Jason Pufahl  07:01

That's a clear definition. I mean, that feels accurate.


Matt Fusaro  07:03

Yeah, for sure. And so you would take something like a timeline that you get out of a forensic activity, maybe include that in the data that you're looking at for your threat hunt.


Steven Maresca  07:12

Right, it's bi-directional.


Matt Fusaro  07:13

It, typically forensics will target very specific systems. It's rare that we end up doing forensics on a really large group of things, that has happened before, but it's rare, threat hunts will take some of the markers that you'd might get out of a forensic activity, and then go search for that in the larger environment.


Steven Maresca  07:34

I have a really good way of making a distinction here. If forensics, forensics is really what you do, in terms of analysis, when you're not otherwise recording the events or the data. So for example, it's it's rare, that all file activity logging occurs in organizations, it's very intensive. You don't know who touched the file, who modified it, who created it, who wrote it, that stuff tends to not occur. It's very expensive, and intensive in systems. So you might take data from a threat hunting platform, and then pass it over to file system forensics to figure out an exact pattern of activity that you can only really divine from looking deeply at a file system.


Jason Pufahl  08:14

Right. So when we're talking about the proactive side of this, and I think, Steve, maybe as a slight segue from what you just described, lots of people don't collect the amount of data that we would love to see in the, if we were in an incident, right. But it's becoming increasingly common that clients have endpoint detection and response platforms in place. You know, they're they're generating alerts with sort of a variety of different severities. What does that look like? Do you consider that threat hunting when you're reviewing that data? Do you consider that, you're probably not an incident at that point, but describe taking a system like that and using the data proactively to reduce risk.


Matt Fusaro  09:03

Yeah, so the way we do it with with our MDR services is that we take, you know, specifically where we work a lot with Defender for Endpoint from Microsoft. And we're lucky because they've got a lot of capabilities in the threat hunting arena. So there's a full query language, called KQL, that you can use to search through all the data that comes back from your EDR. And some of that data would include file actions, network activity, user log ons, log offs, a lot of that stuff that you'd expect out of a full featured EDR. When we get an alert, typically it's it's a kind of an isolated event, we're going to look and see, what it is that happens, probably on a particular endpoint. If we see something in that data that suggests an attacker has, you know, breached, a firewall or something like that. There's an active attacker in the network. We might take some of the data that we see in there, such as you know, file names or process names, network connections that were made, and use that as a threat hunt, or as a set of data for the threat hunt, to see if that's in larger environment, or if anyone else is trying to get to that point of compromise. Right?


Steven Maresca  10:18

That's a particularly important point, in fact, because I want to emphasize just, again, threat hunting is focused on taking single data points and applying them broadly. That's the crux of threat hunting, generally speaking, you may start with a particular system, but it's the analysis that tries to find similar patterns elsewhere, that that that's a really important facet of threat hunting, you may get some of those capabilities of in an EDR platform. They're only really useful and appropriately called threat hunting, when you can make those inferences and leaps to multiple systems for determining patterns,


Matt Fusaro  10:56

Right, typically, the first step of a threat hunt is that you want to create a hypothesis, right? That hypothesis should be driving the whole reason for what you're doing. So an example is kind of what we just talked about, let's say we have an alert on an endpoint, we know that there's a particular IP that's connecting to that system, let's use the hypothesis of anyone that has made a network connection using that IP is compromised, right, and now we go out and search for that. And if we get data back involving that, then we can make a good conclusion saying that these these systems may either be compromised or about to be compromised. Let's see what we can do to actually protect against that.


Jason Pufahl  11:36

Right. And oftentimes, it's taking that data and identifying legitimate activity, right? I mean, that could be, that could very well be the outcome. Somebody escalated privileges for some purpose, but it was intentional. You want to have that communication with our clients to say, we saw something, it's important to take a look, let's validate whether or not there's a risk here, or, we need to spend more time on it. And that's the outcome of this a lot of time. 


Matt Fusaro  12:02

Yeah. A lot of those types of activity where it could be good, it could be bad based on intention, right? You know, is this log on good or not? A lot of those things end up in systems where you have an automated threat hunting system, right? Because that's not something you'd want to do manually all the time, right? So if your technologies are capable of it, either your endpoint systems or your, your SIEM or anything like that, you basically can schedule queries that run against that data to see if we've got hits that say, hey, you know, this stuff was involved, you're now going to have to make a determination whether this is good or bad activity.


Jason Pufahl  12:44

So maybe my last question, we get called for an incident, threat hunters always a component of an incident of an incident for us. We periodically get calls a month after somebody has had an incident, right? And either they've had, maybe they've had failed incident response, or they feel like they have not gotten the conclusion they want out of the work, they did incident response. Can you have can you have a reactive threat hunt? Or do you still consider that then part of the IR activity, even if it's four weeks, six weeks, eight weeks later?


Matt Fusaro  13:19

Yeah, I consider that, you'd still be able to do a threat hunt there. That's definitely one of the triggers to go in to do that. We typically approach those a bit differently. We're not so focused on containment right away, we're not so focused on you know, maybe we're not deploying a bunch of forensic software getting ready to be able to respond if an attacker's moving around. We're here after the fact, we're really looking to gather data, make some determinations, see what we see. And then maybe go back into incident response if we have to. 


Jason Pufahl  13:51

So, maybe validate if the attacker is still resident, and if so, right, restart that IR process. Yeah. So I mean, I think that probably clears it up pretty well. I actually liked the I liked that we segue a bit into forensics, because I think they oftentimes get completed. And I think to separate that out, make some make some good sense,


Steven Maresca  14:12

They're overlapping disciplines is the end of the story, really.


Jason Pufahl  14:15

But I mean, it's fair to say, threat hunting is a fair amount of work. And it's a really ongoing activity, right. So I mean, we get called for it periodically, like as for discrete things, but if you've got security tools, really the review of data that comes from those, it can be really onerous, but that's effectively a proactive, maybe that's your proactive activity. Right? That's your threat hunting.


Steven Maresca  14:37

Yeah. But, by performing threat hunting, or, you know, leveraging a tool of threat hunting ahead of time, you're effectively shifting the burden from crisis mode a little closer to past activity. Just makes everything a little more efficient, gives you better data to operate on and make better decisions is the end of the way of framing it from my standpoint.


Jason Pufahl  14:57

Yeah, there's just, there's a lot of data there to weed through, and most really, seems like most organizations don't have the staff, and really probably the maybe the engineering or analytical capabilities to go through that.


Steven Maresca  15:09

Right. And if you had an incident that had a lot of ambiguity, you might consider installing threat hunting software as sort of a preventative step or an aid for the future to make things a little more effective.


Jason Pufahl  15:20

Sure. Any parting thoughts at all Matt?


Matt Fusaro  15:23

No, I think if you do have a security team, and people who are capable of this, I definitely recommend start involving that process and what you do, you know, for your plan yearly, maybe a couple of times a year, at least start there and look into some into automating some of that as well. Right, to at least get you to a point where you're being a little bit more proactive with all that data that you're gathering, instead of just kind of letting it sit there. And you know, if an alert pops, we'll go take care of it, try to get in front of it a bit more.


Jason Pufahl  15:54

So hopefully, that clears up maybe a little bit of the mystery of IR and threat hunting. I feel like we regularly get calls for both. I think sometimes people don't quite recognize the severe situation they might be in and it might be more of an IR but but definitely look at the data you have, try to be proactive with it. It's a bit of it's, I'd say it's a bit of an art. But it's one you can learn if you do it regularly enough. So as always, hopefully this was informative, hopefully clears up a little bit about those two things are, any questions feel free to reach out to us, we're happy to talk in more depth. Thanks for listening.


16:30

We'd love to hear your feedback. Feel free to get in touch at Vancord on LinkedIn or on Twitter at Vancordsecurity. And remember, stay vigilant, stay resilient. This has been CyberSound.