Episode 58
Listen to this episode on
Episode Transcript
This is CyberSound. Your simplified and fundamentals-focused source for all things cybersecurity, with your hosts, Jason Pufahl, Steven Maresca, and Matt Fusaro.Jason Pufahl 00:13
Welcome to CyberSound. I’m your host, Jason Pufahl. Today, we’re actually joined by three incident handlers. We’ve got Matt Fusaro, Steve Maresca and Suzanne Pare. And we’re going to actually go down the path of doing an incident response sort of tabletop activity. So for those of you who might be familiar with the company, Black Hills Information Security, created a game called Backdoors and Breaches. You can play it with an actual deck, or you can play it with an online version, which is…
This is CyberSound. Your simplified and fundamentals-focused source for all things cybersecurity, with your hosts, Jason Pufahl, Steven Maresca, and Matt Fusaro.Jason Pufahl 00:13
Welcome to CyberSound. I’m your host, Jason Pufahl. Today, we’re actually joined by three incident handlers. We’ve got Matt Fusaro, Steve Maresca and Suzanne Pare. And we’re going to actually go down the path of doing an incident response sort of tabletop activity. So for those of you who might be familiar with the company, Black Hills Information Security, created a game called Backdoors and Breaches. You can play it with an actual deck, or you can play it with an online version, which is what we’re going to do here. So we’re going to set a scenario up, we’re going to spend about 15 minutes perhaps going through the mock scenario, playing the game, really with the intention of simply trying to give people a sense of what it’s like to have an incident occur, talk through some of the different scenarios relative to incident responses and containment. And then potentially the resolution of the incident, we’re only going to spend 15 minutes on this. So we may or may not actually succeed in containing or restoring all of our infrastructure, depending on what happens, I’ll lay out a scenario here, and then allow people to sort of talk through it. But just to give people a sense, there’s really two tiers of cards here. The top tier is specifically the scenario. And the goal of our incident responders is to identify what the initial compromise is, how the attacker pivoted and escalated throughout the environment, maybe how they stole data. So their command and control and exfiltration cards. And then potentially, if they can, if their crack handlers, figure out what their persistence mechanism was, as well. They’re allowed to use their vetted procedures. So they have four what are called established procedures, those are essentially the things that theoretically, they’ve got well written procedures for, maybe they’ve even practiced them or, or had to use them in the past, then they’ve got other procedures, which might be technologies that they’ve recently implemented, perhaps they don’t have as fully built out, or maybe not as comfortable with. So they will get essentially multipliers, rolling multipliers, if they want to use their established procedures. So I’m going to randomly assign the number three, to network threat hunting. And the number two, just because I love it, security information and event management, because that is something all security professionals love. So with that, let’s set the stage of our scenario. And this is going to be a complicated one. The University, which already has distributed IT, has experienced some erratic behavior, you’ve got people complaining that they came in, and their computers were blue screened, you got some folks actually saying that their computers are on but behaving erratically. And unfortunately, we’re not getting a lot more details than that, we’re just getting a lot of reports from a lot of different departments, that things aren’t operating as they’re used to. And unfortunately, because you’re users, you’re not getting really high quality information, right, things like, I can’t get to a folder that I typically use to start my day, my computer’s behaving funny. That’s always a useful technical one. And you know, the central IT departments trying to figure it out. They have some discs still, that they’ve got hanging around, some of these older departments. So you’ve got a real variety now of, of issues coming in. They’re coming to the central helpdesk, and they’re doing their best to convey to IT people, they’re probably calling the network folks and trying to tell them that there are problems. So knowing that you’ve got a variety of things that you can choose from here, what do you guys think you want to start with?Steven Maresca 04:22
So we have options available, we have Endpoint Analysis, which is, you know, related to antivirus and so forth. We have Security Information Event Analysis, SIEM, Log Analysis, User and Entity Behavior Analysis, which is interesting in that it looks for behavioral anomalies, anomalies with users, and we have Network Threat Hunting, which is the, you know, top four tiers. I’m inclined to go toward the Threat Hunting because we have a lot of reports, none of them are specific. Seems like it’s broad.
Expand Transcript
Matt Fusaro 04:54
Yeah, I mean, I think that’s probably a good one. So I think we can go in two directions here, either threat hunting or SIEM, right? Ideally, we have some type of alerts that have popped. If we’ve got this many people that have problems, there’s an alert somewhere, I would hope, so I’d say one of the two of those would be good, I think.
Steven Maresca 05:14
You’ve convinced me. Let’s try for that one. I think we have to roll the dice now. What do you think Suzanne?
Suzanne Pare 05:20
I would go with the threat hunting.
Steven Maresca 05:22
Oh, we rolled a nine out of one to 20. So we’ve failed. That means our attempt to do threat hunting did not succeed.
Matt Fusaro 05:22
Let’s do it.
Jason Pufahl 05:36
Actually, you chose threat hunting, which got a plus three multiplier. So you in fact, got a 12. So you had success because you had good documented procedures and you’re really adept at that sort of that technical capability.
Matt Fusaro 05:52
See, we’re awesome.
Steven Maresca 05:53
Wow. Good for us. Success.
Jason Pufahl 05:55
Good for you for writing things down. Alright, so I’m gonna reveal a card then, and let’s see what we’ve learned.
Matt Fusaro 06:03
Alright, this is the initial compromise.
Jason Pufahl 06:07
So the initial, somebody brought their own device. So because it’s higher ed, and we’re and it’s an incredibly permissive environment, we won’t blame it on faculty, right? It could be anybody, but somebody came in with their own exploited device, it appears. They’re suggesting that maybe you turn to the Firewall or Network Threat Hunting, which I think actually gave you the clarity you needed to find this initially. So now you know that somebody brought an exploited device in.
Steven Maresca 06:36
So that can mean a lot of different things as the next step, I suppose. What do we do next? If it’s an exploited device, I want to know what it touched. So I think returning perhaps to what Matt suggested before; Log Analysis?
Matt Fusaro 06:52
Yeah, I think so we can do that. Or we can look at the endpoint itself, right.
Steven Maresca 06:58
I suppose we know what it is by now, yeah.
Suzanne Pare 07:01
What about the User and Entity Behavior and Analytics? Should we not be considering that?
Matt Fusaro 07:09
Yeah, we probably want to look at that too. Because we wanna know what users were there, right, just because we, because we have just the device right?
Steven Maresca 07:18
I think so.
Matt Fusaro 07:19
We don’t know who was using it. We assume it was just that person.
Steven Maresca 07:22
I agree. May not be, we might as well find out.
Jason Pufahl 07:29
What do you guys decide, which one is it?
Matt Fusaro 07:32
I mean, so far our established procedures have worked well, let’s go with it, UEBA.
Jason Pufahl 07:38
UEBA, alright, let’s do a roll, see what we end up with. You guys are crack analysts.
Steven Maresca 07:48
We rolled a 15, which is extremely successful. So what does that mean we’ve gained?
Jason Pufahl 07:53
So you know, because you guys are doing so well, do you want to display your pivot and escalate your C2 or your persistence guard?
Steven Maresca 08:04
I think pivot and escalate makes sense to me.
Jason Pufahl 08:06
Just go in order.
Steven Maresca 08:12
These attackers are wily, so they’ve revealed connected identities and access rules in Active Directory. So they know what their compromised user and device can access and what data might be reachable by this point.
Suzanne Pare 08:27
Just unplug everything right now.
Matt Fusaro 08:33
Yeah, unfortunately, we walk into a lot of that.
Steven Maresca 08:35
We do have that as an option. But I’m not sure that it’s, I think the horse is out of the barn from the perspective of isolation.
Matt Fusaro 08:44
Yeah, they’ve probably compromised another account at this point.
Steven Maresca 08:47
So perhaps SIEM is our next step?
Matt Fusaro 09:00
Yeah. I mean, we’re gonna have to look around and see where those users might have been and what data was touched.
Jason Pufahl 09:07
You guys are, you know, unfortunately, taking just too much time to make your decisions. So we’re going to inject a new scenario.
Matt Fusaro 09:17
Oh, we get more capability.
Jason Pufahl 09:20
By delaying, you actually improved your chances of winning. I’m doing a new injector, I don’t really care for this one. Oh, no. You got data that’s been uploaded to Pastebin. So now you have I think you have clearly, you’re starting to clearly see that there is data exfiltration happening, I think now, as a result of that, of course, executives at the institution are getting more concerned because they don’t know if there’s going to be data breach costs in the future. They have no idea what the data is. You might have possibly had some good logging. So maybe you can resolve that.
Steven Maresca 09:52
And the additional little twist is that the external entities are now notifying the organization and the media.
Matt Fusaro 10:01
Time for Crisis Management.
Steven Maresca 10:03
I think so. Yeah, Crisis Management is a reasonable thing to invoke at this point. We do have that as an other alternative procedure, so we might as well give it a shot. One could argue that it’s perhaps a little late, but I think it’s time.
Jason Pufahl 10:17
Alright, well, let’s see if your luck holds.
Matt Fusaro 10:22
Oh, Crisis Management failed.
Jason Pufahl 10:26
So what does that mean? Does that mean just nobody knows actually what to do from here, then? Executive leadership has not gone through a tabletop of any sort.
Suzanne Pare 10:36
Have we done the Firewall log review? Have we done that? I mean,
Matt Fusaro 10:42
We have not done that yet. Although I would hope, I would hope that our firewall is sending information into our SIEM. So that would be ideal.
Steven Maresca 10:51
It’s time to play the SIEM card, I think. Let’s roll again. Oh, fail.
Jason Pufahl 11:03
Even with the multiplier.
Matt Fusaro 11:05
Yeah, we’re in a bad spot.
Jason Pufahl 11:07
You had a good run at the beginning. But now you’re really struggling.
Matt Fusaro 11:09
You had no Crisis Management, our logs are no good.
Jason Pufahl 11:15
So maybe taking a step back, that’s reality for a lot of these incidents, for sure. You roll in it with no data. And you’re trying to guess, and people aren’t sure how to react across the organization.
Steven Maresca 11:28
Or attackers have cleared the logs, which is certainly not something we’ve avoided, with some incidents.
Jason Pufahl 11:34
Alright.
Matt Fusaro 11:36
So what do we do now? Well, don’t have a lot of options left here.
Steven Maresca 11:45
I think we’re at the server analysis or isolation side of the equation. We need to buy time or just presume that things are compromised enough that we need to rebuild.
Matt Fusaro 11:55
Yeah, I mean, we don’t want more data leaving the organization. So I think it’s probably time to isolate at a pretty high level, right? Because we don’t even know where this stuff has gone. So we don’t know what data was pulled out in addition to what’s been put on Pastebin already. So, that’s a rough one.
Steven Maresca 12:15
Alright, so roll away.
Jason Pufahl 12:17
Let’s roll away and see what we get. And because I’m feeling pretty good, I know that there’s good network capabilities, I think we just gave a bonus to isolation. So let’s see where we end up. Another failure, you can’t even isolate. So you rolled a collective 7.
Steven Maresca 12:36
We can try procedures again, that is permitted. How many turns do we have left? There is a maximum here, we have four turns remaining. We have to succeed, eventually,
Jason Pufahl 12:45
We might finish this game in the time that we have allotted.
Matt Fusaro 12:47
I think you’re back to Threat Hunting.
Jason Pufahl 12:47
So let’s do a little Threat Hunting.
Matt Fusaro 12:56
We were good at that last time.
Steven Maresca 13:03
Failure.
Suzanne Pare 13:04
I think there’s something wrong with your computer.
Jason Pufahl 13:09
Well, just because we’re good sports, let’s see what happens with a new inject card. So you did get a little extra information, you had a honeypot deployed. And that does give you some information. for what that’s worth.
Matt Fusaro 13:27
There we go. Looks like we can turn over the pivot card, right.
Jason Pufahl 13:33
Alright, let’s do it.
Matt Fusaro 13:36
Oh, wait, I think we already did. So, we got no new information,
Jason Pufahl 13:40
No new information. Alright, one more inject.
Steven Maresca 13:44
So the log analyst returns from training. So we get plus two for any roles that involve log related actions.
Jason Pufahl 13:57
If you actually have any data.
Matt Fusaro 13:58
Yeah, yeah. I guess let’s use them then.
Jason Pufahl 14:05
So what are we doing? You’ve got some more established procedures in there. Is there anything you want to try?
Matt Fusaro 14:14
Well, we can’t really use Endpoint Analysis, right, and I don’t think that that’s going to tell us too much, because we don’t know where they’ve been. We can start with that initial, right, the initial endpoint, those under the first one, but I don’t know that we’ll get much data out of that. We might know a couple places to start looking. So I mean, I guess it could be somewhat useful, but we really want to, we really, really want to use our logs at this point. We need a bigger picture view of where the logs are.
Steven Maresca 14:42
I think so. So honestly, I think we need to retry that Log Analysis procedure.
Matt Fusaro 14:48
The Firewall log or the SIEM?
Steven Maresca 14:52
Fair question.
Matt Fusaro 14:55
I mean, I like our SIEM.
Steven Maresca 14:56
So let’s try the SIEM.
Jason Pufahl 14:57
I mean, your SIEM analyst did just get back the training, it seems like the luck is on your side now.
Steven Maresca 15:04
And it’s a plus two, on top of plus two. So hopefully we’ll,
Jason Pufahl 15:08
Alright, so let’s see what we get. We’ll roll one more time. If you get the SIEM, I think maybe I’ll give you some success overall.
Steven Maresca 15:18
Oh, very successful.
Jason Pufahl 15:20
Now that he’s trained, he’s killing it.
Matt Fusaro 15:24
He’ll forget it all in a week.
Jason Pufahl 15:25
But it doesn’t matter. He knows it now. Alright, well, we know what data was stolen. Let’s see what happened in the exfilt card here. The attackers use domain fronting to bounce the traffic off of legitimate systems. What does that mean for everybody? Actually just using uncompromised systems to send data or using DNS or clever techniques like that to pull data out?
Steven Maresca 15:56
Well, it does suggest that isolation is a reasonable next step regardless because you know, it’s difficult to clamp down on otherwise.
Jason Pufahl 16:05
So you’re going to try isolation, again, in spite of the fact that you had no capabilities of doing it before?
Matt Fusaro 16:10
I think this time, we just pulled the plug on the internet.
Jason Pufahl 16:14
Alright, let’s give it a go. That’s the real debate. And so in fairness, we’ve done that in the past. Right, this option when you’re struggling.
Steven Maresca 16:24
With a plus three, I think we did succeed there,
Jason Pufahl 16:27
Fully isolated. Let’s see what we have.
Steven Maresca 16:33
Crafty, we’ve discovered that the attackers used a malicious driver in the operating system for actual persistence, which explains the pervasive access and difficulty in finding things in this environment. But now we know what to remove.
Matt Fusaro 16:54
So I think we won, somehow.
Jason Pufahl 16:59
Matt, it’s not about winning and losing, it’s all about learning something.
Matt Fusaro 17:04
Yeah, I don’t think we ever finished an incident and said, hey, we won!
Jason Pufahl 17:08
in fact, right now you’ve left the client with a completely severed network, they actually can’t work at all. But we have, we have turned over all of the cards.
Matt Fusaro 17:18
As silly as this whole process was, it’s more realistic than it should be. There’s been plenty of times where we’ve had to go and do things like isolation and weren’t able to do it or the tool to do it wasn’t working properly, etc.
Steven Maresca 17:34
Or, this is where we get called, right? We’ve figured out some things and don’t know where to go to clean up, or it’s too big a job to clean up, how can you help us?
Matt Fusaro 17:45
Yeah, and don’t discount the Crisis Management failing. A lot of times, we end up coming in there being the Crisis Management, or at least getting the right team together for Crisis Management. Most organizations don’t know how to pull in the right legal team and insurance and all the people you need to talk to you, quite honestly, just being a therapist for a little while, right?
Steven Maresca 18:06
We craft more communications on this front than you’d imagine in a technical incident handling role, right.
Jason Pufahl 18:14
So our recommendation is, you know, this game is kind of fun. And it is certainly easy to do over Zoom, the web-based version makes it really nice. But at the root, right, all we’re really trying to say is practice a little bit your incident response procedures. If you’ve got tools, make sure you got the familiarization of how to use them. Write down your processes. But act this stuff out a little bit, you know, this was meant to be fun. Actually, I think I’m not sure how long we went. But we we did pretty good rolling through a set of technical controls or responses to this. Again, it’s Backdoors and Breaches, you can go to play Backdoorsandbreaches.com. Certainly a shout out to Black Hills Information Security for putting a game like this together. Because I think it does try to take a really serious topic and make it enjoyable for a group of folks to get together and play. If you’ve got the physical game, you can actually pull out some of the cards or add cards depending on the audience that you’re trying to sort of train. So you. As always, if anybody’s interested in talking about the incident response tabletops or just generally incident response overall, or maybe you want to play a game with us. Hit us up on LinkedIn at Vancord. We’re happy to continue the discussion. And of course, as always, we hope people got enjoyment and value out of this podcast. Thanks for listening.
19:37
We’d love to hear your feedback. Feel free to get in touch at Vancord on LinkedIn or on Twitter at Vancordsecurity. And remember, stay vigilant, stay resilient. This has been CyberSound.
