[Music] hello welcome to another episode of the virtual ninja show today I'm super excited because we have Michael back Michael I know probably everybody knows you from the show but please introduce yourself absolutely thanks uh thanks haup so my name is Mike Malone I'm a principal security researcher on the Microsoft vender experts for xdr team so yeah and you're my favorite threat hunter that I could have asked to join that show obviously so Michael we're going to talk about yes how does a day
in a life of a sock analyst look like how would you explain this to me absolutely so the challenge with uh working at a sock is you've got a lot of competing priorities there's always something new to look at uh the the challenge is figuring out what to prioritize and to be most effective from a time management perspective from a security impact perspective uh we also want to look at you know the organization security and then if possible you want to also focus on the continuous Improvement of
the entire organization security operations system so in other words contributing back into intellectual property for your organization that helps you hunt things more efficiently more effectively in the you know in the future so there's a whole bunch of different things you got to kind of pay attention to so even though I might assume that it there is of course a few variables that are different from customer to customer maybe even segment to segment size to size but there's also a few things t
hat should be done and are important for any size of organization is that correct absolutely exactly what you identified is the reason that threat hunting has to exist regardless so uh we can detect all kinds of great stuff with the defender stack um but every organization is different you have different priorities you have different uh different Assets in your organization that are more important than others and that the threats and the normal operation of your business is going to be different
from other businesses so threat hunting is really taking the security domain knowledge that you've got in your head and applying it to your unique organization's needs sometimes you got to expose something a little bit more and you need to tune your detections around that particular type of asset sometimes for organization something is even more important so you need to tune your detections in that manner and that's exactly the reason that we do this this this threat hunting over hunting and an
d customized vulnerability management and this is also something that you do on a daily basis right so you work on the defender uh expert for xdr team absolutely so the defender experts for xdr team and the devor experts for hunting team both do over hunting over the customer environment so there's a couple of challenges you run into with threat hunting uh the first one is time of that the detection is published sometimes sometimes something is brand new nobody has seen the domain before you mig
ht be the very first or one of the first targets using this particular technique so it takes a second for detection logic to catch up with it uh this is the reason that uh defend directors for hunting exists a lot is is to cover that Gap so I kind of retr hunt over the environment and look for latent threats that may not have been detected at the time that they occurred because the signature may not be ready yet for that particular threat uh there's also a lot of General over hunting recommendat
ions things that you can't really quantify as an alert but you kind of put a bunch of pieces together and you say okay this is a web server uh we've got a domain ab and credential being exposed to it we probably want to tell the customer that you don't want to use highly privileged accounts on a on a web exposed asset or maybe we identify overprivileged scenarios where where a service could probably run with less permission and and the way it's running is a potential risk to the organization so
these are some of the things that we try to highlight in Defender xdr to give the customer some insight and the ability to move forward confidently uh in in their operations so of course you just mentioned it right so there is a device that might be a domain controller that might be a web server there might be credentials like accessing it that have high privileges is that something that the portal gives me right away or is this already something that you would say Hey you see there is I don't k
now a vulnerability on a web server so now you need to go hunt and search for more things that are important to understand before you do something absolutely yeah so the portal gives you a bunch of great stuff in Defender vulnerability management especially let's go ahead and pivot over to the portal let to look around inside see we can find so there's a few different things you want to kind of pay attention to from a especially from a vulnerability and configuration manag perspective uh Defende
r vulnerability management capability we'll go over here to the dashboard gives you a really good oversight into overall vulnerabilities that are out there and you're kind of comparing really two main things from this perspective one is are things configured properly ie able to defend against a threat because you've got your best security configuration for that particular application that operating system Etc and there's also the weaknesses or vulnerabilities in software which imply that you kno
w if if an adversary was to identify this vulnerability and have the ability to access it they could potentially circumvent it regardless of what control you put in place and when I say that they have to still have the ability to touch the software obviously in order to exploit it and it has to be the right kind of vulnerability so let's go ahead and take a look under weaknesses real quick now mind you this is a small lab environment it's meant for demonstration this is not really a great repres
entation of a 10,000 20,000 seat organization uh in this small lab environment we've got 412 vulnerabilities out there and we've got a couple different ways that we can sort them we can look at them based on if there's a known exploit there's public available if there's an adversary that's currently using it we might see this little Target get lit up uh the other thing to pay attention to is the CVSs with the common vulnerability scoring system which tells you how bad it is and this comes from a
number of different factors such as how is it accessible in other words is it locally exploitable is a network exploitable what permissions does it yield when it's exploited Etc now there's another layer of that you might want to dig into when you're looking at this which is how relevant is this particular vulnerability to my specific scenario so for example if I have a web exploitable vulnerability on a device that is internal only has no internet access and it's only uh because of firewalls r
outing or vpns or whatnot it's very limited from an accessibility perspective to maybe only people who are inside the network you might want to rank that one lower than a similar vulnerability that's on something that's public facing or maybe that vulnerability on a system that has highly sensitive data or has regular exposure to admin credentials so maybe like a jump server domain controller somewhere you're doing authentication somewhere where authorization is granted inside your Enterprise so
one of the neat things you can do I'm going to pop over to Advanced hunting real quick you can actually pivot this based on the information inside your organization so for example uh we can take a look at vulnerabilities uh that are on internet facing devices for example so by taking the device info table and pivoting on where is internet facing it's going to let me know that these particular devices are facing the internet and then I'm going bring up the TVM software vulnerabilities to figure
out what vulnerabilities are on those devices and instead of looking at 412 off the top of my head I'm now looking at 26 and you can use this with other pivots as well looking for powerful admin accounts maybe sensitive information on these on these particular devices and use that to kind of Stack rank what you say is important to your particular organization from a vulnerability management perspective wow okay yeah and of course you can go wild and combine and join the different tables and and
information that you want to get as a result and of course this is now you know you look into vulnerabilities misconfiguration and what's really important for your organization but there's the other side which is you have an alert or you have an incident um so how do you go with that because yes it's a lab environment but even in that I mean I would be scared looking at it because there's a lot of incidents there's a lot of alerts how do you prioritize those and and and how do you yeah navigate
there sure so there's a couple different angles so let's take a step over to the in product alerts we provide a pretty good picture of what the severity of a particular event is from the alert level and that trickles up into the incident level and gives you the ability to prioritize these types of activities so if we can see we have a multi-stage activity we've ranked this one high we've also got the defender experts tag on there which means this is a Defender experts customer and we're going to
start taking a look at and in some cases responding to the threats automatically depending on the configuration of the tenant obviously first things first in normal day-to-day sock work this is your priority because these are the threats that are knocking at your door these are the adversaries who are trying to fish their way into your organization steal credentials this might be an initial footprint this might be an exploitation of one of the vulnerabilities that we just talked about previousl
y in vulnerability management so this kind of takes priority over everything uh the neat thing here though is that going back to this customization for your particular environment these are the outof the-box capabilities you're also going to have some stuff that you're going to Define that's specific to your organization and this is kind of that whole security continuous Improvement cycle right now I would say focus on the the alerts and incidents but then when you get to the point where the ale
rt incidents that remain are either very low priority are all resolved or you have you're maybe you have some free resources in your security operations center kind of the next thing to do is kind of over hunting and looking over your organization looking for things that are maybe not product detections because sometimes the concepts um that we would be looking for may be uh tailored to a specific organization their mission their specific identities Etc oh you have specific threat intelligence u
m that is for that you use that you want to include right the important thing when you're starting to perform threat hunting inside your organization you want to kind of focus on ensuring you have a good return on investment you want to make sure your time is used effectively because there's a ton of data inside Defender um you got to make sure you're getting you're getting your money's worth if you will out of your time you're spending performance task so I have this concept I call the detectio
n funnel which kind of helps you prioritize what types of detections to perform in what order based on your free time and based on what level you're currently at in your organization the highest Fidelity the best return investment is going to be some sort of a validated indicator of attack or indicator of compromise this is going to be either a product alert this might be a custom detection maybe a sentinel analytic rule that you guys wrote in house based on previous events that you've observed
inside your organization so those are all always where you want to spend your time first close that CU of anything that's uh high medium severity even and then obviously over hunt over your lows and information will make sure you got good coverage for those type of things the next one in the funnel is going to be thread intelligence this is not to imply that threat intelligence is lower value but it's going to include a lot of things that you may not have observed inside your organization this c
ould be either third-party threat intelligence first party from your own investigations Defender threat intelligence uh but this is going to be the IPS ashes domain names URLs that you're getting fed from whatever feeds you you currently subscribe to or have available below that we start getting into the Customs where you're starting to develop new intellectual property for your organization so pattern-based detections Behavior based detections anomaly based detections so a pattern-based detecti
on is looking for something sort of suspicious uh it may not you may not know it's malicious but for example if you wanted to identify a specific maybe maybe you want look for uh applications like shell applications like maybe Powershell or something that's going to a suspicious looking website or maybe you want to Pivot on what you know on on what types of like a certain set of websites you want to dig into to see if there's anything potentially malicious associated with that activity so that w
ould be a potential pattern where you're trying to look for in like the launch string or in the network connection maybe this process paired with this specific destination maybe the U maybe the geolocation of a remote IP address might be of interest to your organization that would a pattern behavioral is sort of focusing on pairing together a known and looking for things that are unknown so for example this process always Network connects to these particular destinations this user always perform
s this particular action um you want to if you have a baseline you can work off of you can filter your results so you can focus on and filtering out all the knowns and focus on the unknowns the last lowest tier of the detection funnel is the anomaly tier and anomaly tier is basically everything else this is looking for things that by statistically never happen to your organization this this process never launches this this this condition never happens this never happens at this hour of the day t
hat is your lower return investment scenarios because it's going to it's while it's the most fun and you will find some really interesting stuff you've got the problem of the of the long tail they call it in statistics which is there are a ton of anomalies in any organization of any size so Michael this is amazing this funnel can you show us a few example of how you actually go down that funnel yeah let's pop into Advance hunting and check it out uh I'm going to skip over our IA and ioc detectio
ns and threat intelligence detections because those are pretty straightforward you're going be looking for a hash and IP address you're going to be looking at the alerts that came out of the box let's focus on the stuff that's going to be a little more custom for the organization like starting at like a pattern detection so first off let's say the hypothesis is I know I've seen evidence of malicious software doing direct connections out to GitHub so we want to take a look at what processes in my
environment that are not web browser processes went to GitHub this might be an automated way of downloading something and executing in the system so in this case we're going to go ahead and take a look at the network connections where the process is not Edge Chrome Firefox Opera also we know that git the software will go out to GitHub so that's we're not too worried about that and there's an updator for some software that will go out to GitHub so we'll filter those out look to see where the rem
ote URLs has GitHub in it and when we kick this off we're going to find out that we in fact have an interesting Powershell script here so this device AV o r i a z has powershell.exe running uh a script C scripts webn PS1 that goes out to GitHub on a regular basis now the question becomes is this malicious or is this benign at this point in time we don't really have an indication that this is this is a malicious or benign we're going to need to dig into that c scripts webn.com to figure out what
it is but ultimately you may find something that's out there but notice that we got a pretty decent return so in this case we only brought back nine rows on seven days worth of data in this lab environment obviously it's larger in a production environment um and you're going to have a lot more noise but you're if you're smart with the way you write your pattern detection queries you shouldn't have a ton of rows if you get a ton of rows in this type of query you probably need to add more filters
MH so you would basically now of course continue the investigation as you said you would go into the script and you would look what the script is doing um but you would not have seen for now an alert on this specific activity because usually Powershell accessing GitHub is not malicious perc right so it's for you something that you would like to see or like understand for you this organization exactly yeah so the thing is you have to so this the whole point of this is to custom tune Things based
on how your organization operates uh if we focus solely on on if we can't write a product alert saying Powershell going to GitHub is bad because there's a lot of legitimate reasons that Powershell can go to GitHub and every organization is different if we made an alert on every single one of those people would never get done with their queue and they would probably be very angry at the signal to noise ratio of the things that we produce as a product and that's the fundamental problem you get int
o with threat hunting is you are kind of tuning it to your organization you may have a policy that nothing you know the Powershell can't go to any web address for example depending on how strict how secure your organization is and that may be a violation of that and that may be the hypothesis that led you to run that query yeah so let's go ahead and take a look at the bottom of the funnel now where we're going to have a lower return on investment and kind of dig into some techniques you can use
around that I'm going to take a look at the very most basic type of anomaly detection which is like a statistical anomaly detection so in this case we're going to look at processes uh and we're going to figure out how many different devices ran a specific process command line so how many different devices ran this one specific command line per command line we're going to then filter it out we have to set a threshold because we don't want to see every device ran this not really worried about that
you're looking for something as a statistical anomy right now so we're going to reduce that to only command lines that ran on less than five different devices inside the organization bring it back with the original data and then sort it based on how many machines ran that command in the last seven days right now so we'll go and run this this is going to take a little bit longer because we are doing a we're do a join here and notice a difference here this time we got 10,000 results back which is
the max you're going to get in your Advanced hunting web UI and this is kind of showing you the difference in return on investment the further you go down that detection funnel the lower the return investment so and let me show you an example here so note that these are all by the way these commands each ran only on one device inside the organization so these are all technically anomalies and some of them could be interesting some of them not that I was talking about before the tail problem let
's just go ahead and visualize it real quick so the tail problem is showing you we're going to take uh this is let me do switches to actually process command line so it matches our query above and we're going go ahead and take a look and see what the distribution is of the number of one offs from two offs three offs and four offs so we can see we have 19 machines looks like inside the environment but look at the sheer size of how many things only existed on one machine in seven days and this is
why you have to be super efficient in fact we can just do uh with Y axis log just so we can kind of see a better distribution here we'll do a log axis on this and even even then we still can't see because there's only 252 where 252 instances where two machines ran the same command line and that's kind of sh showing you the challenge you've got when you're trying to do this threat hunting is it is in the anomaly in the bottom of the funnel it is inherently lower Roi than doing something like beha
vioral pattern signatur like IA ioc type detections that's why it's luckily at the very end of it exactly you want to make sure you focus on the things that are going to be tied to some known threat actor TTP uh you going to have something to tied to threat intelligence focus on that before you start getting into the pure anomaly type of detections because you're going to your time's going to be used a lot more wisely yeah I mean even though it looks like fun and um going there but I think as yo
u say like it's also the time like how much time do you have and you need to focus on the most important things um very much on top and for this this is something that you might do once a week if you have some free hours you have a few free hours you know uh the queue is quiet maybe it's a Friday where we tend to see less alerts on Fridays I'm not sure why but the uh you know maybe that's a Friday type task where you have you know you've got your vulnerability management stuff in check you've go
t your daily alerts and investigations things are kind of coming along no big problem there just kind of peek around in the environment and see what you see what you find out there and that might be the basis of a new detection how do you use a lot of advanced hunting um you use the Tool uh we also you mentioned um analytics rules and Sentinel so how are all these things coming together for you yeah so this is all part of uh having a continuous improvement process so we've got a concept that we
we like to talk about called hypothesis driven threat hunting and basically what you have is it's sort of like applying the scientific method to the thread hunting domain you got you start out with the hypothesis what is the likelihood that a specific threat actor is in the environment what's the likelihood that something is being abused you got to come into it with a question of some sort and then you come up with one or more different queries on the data that you have to prove or disprove that
condition so when you're looking for storm 1 two 34 5 in your environment you need to have a set of indicators that storm 1 2 3 45 might be there if you want to know what is the possibility that somebody's abusing Powershell to to act to to perform remote command and control you're going to come up with a number of different queries that can ultimately be used to prove or disprove it and depending on what you find the you want to take that and roll that right back into your custom detections yo
ur analytic rules uh all your uh your notebooks perhaps to try and investigate these scenarios uh in the future more efficiently maybe even a create automatic alerts that way in the future you don't have to go through that process anymore and ultimately what you end up doing is you build up this cache of really good detections that are custom tuned for the things that your particular organization is interested in the topic was basically the day in a life of a sock analyst of course you go down t
he funnel so it can change any day there is no this is always what you do in this day right I mean it can change any day plus you might also consider looking at the vulnerability dashboard that we showed before and there's something super important and it might be more important than solving the second incident in the queue so how do you go there as a customer I'm just trying to understand for myself wow have no clue about threat hunting but imagine someone would hire me as a threat Hunter and t
hey sit me in front of a console and say fix it make a make it good so is is it um you there is no really like this those are the daily tasks and those are the weekly tasks is it so complicated because it depends and it changes so it depends a lot on how much free time you have in your organization and it depends on your how advanced an organization you are how proactive the organization is from a security perspective now organ some organizations are going to have a lot of noise they're going to
onboard Defender they're going to get tons and tons of alerts this is where you get into like needing suppression is a common scenario right are these false pauses are these benign pauses I.E sure the detection is correct but we don't care about it in this scenario because maybe this is a vulnerability scander inside the Enterprise so trying to tune and get to high Fidel detections for the stuff that's already in the Box the next I mean the other thing you're it's uh you also have as I mentione
d the vulnerability management side of the house so you want to make sure you keep your vulnerabilities under control as and you want to do so relative to the relative risk of that particular asset that is vulnerable uh as well as keeping an eye on your overall road map from from a from a security hardening perspective the alert tuning and suppression side of the house is pretty much a day-to-day task because you're going to come across an alert you're going to say I've seen this three times it'
s always a false positive let's go ahead and suppress this one uh from a vulnerability management perspective some organizations have a dedicated vulnerability management team other ones are a little bit smaller organizations and everybody kind of does the like maybe this the sock team does a management in-house uh so depending on your organization uh I mean I would say weekly to monthly is a is probably a good good Cadence for for keeping an eye on the vulnerabilities and I say that only becaus
e we have a monthly patching cycle and most organizations have a release cycle that's going to be you know it's going to it's not going to be every single day new codes coming out new vulnerabilities will be coming out and you should probably keep breast of anything that that that shows up as public exploitable or anything that shows up as having a a known adversary exploiting a specific vulnerability uh that's a different story so obviously keeping your eye on the news and rep prioritizing base
d on that is going to be super important the other thing sort of like working on the uh on the detection side of the house is kind of what you do when you have that free time after you've beaten the queue down is you don't have a whole bunch of active alerts in the queue you have a plan to address the critical vulnerabilities of the organization that's when you start really having the time to kind of drill into that uh critting that intellectual property Automation and hopefully getting into aut
omating as much as possible the handling of these investigations inside the inside the Enterprise so I think it's also something that's it's a circle like right it it's I I would assume yes of course keep up with the news but not just the news what's currently attacking the world but also product news maybe I just do like a little amplification aka.ms Defender news so you'll learn what are the latest features that you can use that can help you get your job faster done but also um maybe for somet
hing that you have a I don't know let's say a a a workbook written WR or something that is not necessary in the end or there is new features that you might want to configure from a security configuration perspective um that I don't know if they maybe not turned on by default or something so that's basically the yes you need to continuously look into yes incident alerts but also the whole set of configuration and and features that there right to keep them secure yeah as new features come out you
want to make sure we're we're always keeping a breast of the latest uh latest capabilities of the product uh we you know we we're shipping new uh new new features to help secure uh devices every release should say so you want to make sure you're aware of those and always and the nice thing is is our secure our secure score will help reflect that help a customer prioritize that uh so that they can focus on the ones that we're recommending the most based on their current situation yeah and you men
tion it right so if there is a high severity alert and you go into it and you look at the yeah an incident you go into the alerts they will tell you that this is related because of a cve on a device um so this is already part of it but then if you have some spare time um check out even if it's of course on a daily basis um the vulnerabilities the weaknesses the dashboard how you doing are you're going down or you're going up um so that's why we put those visuals in there to actually show someone
um the impact of the work that they're doing on a daily basis what is the last advice that you can give to people that might just start or people that are more experienced but you know it's it's a lot to do um yeah so what's your advice absolutely so there's uh a few things so obviously time management is kind of a key theme on here make sure you focus on things that are going to return for your organization use use the defender Autobox alerts the the defender vulnerability management capabilit
ies to identify risks out there but also make sure you kind of put your own spin on it relative to your organization's specific needs and and from from a security perspective but at the same time make sure you set some time aside maybe a few hours a week to kind of look over the environment look for things that are out of sorts they're abnormal relative to the environment so you want to look for those anomalies because ultimately the more standardized an environment is the higher Fidelity your y
our alerts and your the more efficient your sock is going to be if you've got a bunch of random stuff you want you want to try and tame that and make things as consistent as possible if you want to really get into into threat hunting and stuff definitely check out I've got a series from back when at aka.ms tring the adversary so if you want to get into some of the kql we were talking about in this there's a we have a great four-part uh uh part series on that uh and obviously stay tuned to the ni
nja show for the latest awesome product things that are coming out as well thank you thank you so much Michael it was a pleasure having you back and um I hope I will have you again on the show we can always talk about things I know but you're busy hunting the bad guys um but for now thank you very much Michael and yeah thanks everyone for watching see you [Music] soon
Comments