CrowdStrike Sets Speed Benchmark with Fastest-Ever Threat Detection in MITRE Engenuity ATT&CK® Evaluations: Managed Services, Round 2

Security teams must outpace increasingly fast and sophisticated adversaries to stay ahead. In the most recent closed-book MITRE Engenuity ATT&CK® Evaluations: Managed Services, the CrowdStrike Falcon® platform once again demonstrated it stands alone in its ability to deliver the speed and efficacy needed to stop breaches.

• CrowdStrike Falcon Complete® managed detection and response (MDR) sets a new speed benchmark, scoring the fastest mean time to detect (MTTD) at just 4 minutes — 6 to 11 times faster than competitive vendors.
• CrowdStrike achieved the highest detection coverage score for the second consecutive year — reporting 42 out of the 43 (98%) adversary techniques.

MITRE’s closed-book evaluation emulated a real-world eCrime attack without giving vendors prior knowledge of the threat scenario — creating the most accurate assessment of a vendor’s capabilities. In this scenario, Falcon was operating in detect-only mode, meaning no automated actions could be taken to kill processes.

MITRE does not rank or rate participants — the following is CrowdStrike’s analysis of the results provided by MITRE Engenuity.

We believe these results clearly demonstrate that the powerful combination of the Falcon platform, CrowdStrike’s elite team of experts and our knowledge of the adversary stands alone in the industry when it comes to stopping breaches. The Falcon platform once again achieved the highest detection coverage and fastest mean time to detect at just 4 minutes — an exceptional performance 6-11x faster than comparative vendors.

Organizations must rigorously evaluate MDR vendors and demand cutting-edge technology, unmatched expertise and proven outcomes. Only a unified approach ensures swift and effective threat detection, investigation and response. This is why the results from MITRE’s latest evaluation should be considered holistically. When reviewing these results, ask yourself: What good is speed if threats are missed? What good are actionable detections if they cannot be trusted or acted on quickly? What good is threat detection if detections happen too slowly to prevent breaches?

Not only did CrowdStrike achieve the highest detection coverage and fastest MTTD of all vendors evaluated, we also generated the highest number of actionable notifications and detections, showcasing our ability to drive superior security outcomes — namely, stopping breaches.

Figure 1. Mean time to detect (MTTD) in MITRE Engenuity ATT&CK Evaluations: Managed Services, Round 2

Unsurpassed Speed and Efficacy in MDR

CrowdStrike Falcon Complete MDR achieved remarkable results in the latest MITRE evaluation, building on our previous success in the MITRE Managed Services, Round 1 and the MITRE Enterprise evaluation. We accurately reported 42 of the 43 (98%) adversary techniques, achieving the highest detection coverage of all vendors evaluated.

The CrowdStrike 2024 Global Threat Report shows the average breakout time for eCrime is dropping rapidly, going from 84 minutes in 2022 to just 62 minutes in 2023. The fastest recorded breakout was just over 2 minutes. This real-world data shows every minute counts when sophisticated adversaries attack. With many cybersecurity solutions, by the time a SOC is aware of an intrusion, it’s too late — the adversary will have already moved on to their objective. Falcon Complete detects the attack immediately, preventing a breach.

Organizations must have confidence in their MDR provider’s ability to swiftly detect and eliminate threats with uncompromising efficiency. This closed-book evaluation, in which no vendor had advance notice of the adversary or their TTPs, accurately simulated a real-world attack and offered a precise assessment of each vendor’s ability to detect and report threats, registering MTTD as a critical metric. Vendor response and remediation were not evaluated.

The Falcon Complete team rapidly correlated intelligence and cross-domain data using the Falcon platform’s rich security telemetry, which encompasses endpoint, identity, cloud workloads, third-party data and integrated threat intelligence. CrowdStrike achieved these objectives with astounding speed and accuracy.

Figure 2. Detection coverage category scores in MITRE Engenuity ATT&CK Evaluations: Managed Services, Round 2

Identifying Sophisticated Tradecraft in MITRE Engenuity ATT&CK Evaluations: Managed Services, Round 2

CrowdStrike’s objectives in the evaluation were to investigate and provide context and analysis of Falcon platform detections in order to establish who the threat actor was, identify the earliest and most recent threat actor activity and determine how they gained access to the systems. We were required to present MITRE evidence, if any, that the threat actor accessed or exfiltrated data and identify potential lateral movement to other systems in the environment.

During the evaluation, we reviewed and monitored Falcon platform detections and relevant telemetry across native endpoint and identity data, and network and email third-party telemetry from CrowdStrike Falcon® Next-Gen SIEM, to perform remote triage analysis on impacted systems.

CrowdStrike identified that the MITRE ATT&CK Evaluations for Managed Services emulated the behavior of two sophisticated adversaries tracked as STONE PANDA and ALPHA SPIDER by CrowdStrike Counter Adversary Operations (known as menuPass and ALPHV BlackCat by MITRE).

Figure 3. Incident diagram created by Falcon Complete and the CrowdStrike Falcon® Adversary OverWatch team, mapping adversary technique as the attack unfolded.

Learning How the Threat Actor Gained Access

The threat actor was first detected via a remote desktop connection from an unmanaged system, indicating that valid credentials were likely already compromised and used to initiate the attack. Specifically, the credentials for the “kizumi” account were utilized. Using the CrowdStrike Falcon® Identity Threat Protection module, Falcon Complete quickly identified that there were known risks with the account, including shared passwords with another account and a path to a privileged user.

Figure 4. Falcon Identity Threat Protection highlighting risks tied to the kizumi account

Identifying Threat Actor Activity

Once the threat actor had access to the environment, native tooling using certutil.exe was executed to download malicious binaries from threat actor-controlled infrastructure. These binaries were subsequently sideloaded into the legitimate application Notepad++ and identified to match Sigloader and QuasarRAT malware, confirming the STONE PANDA attribution.

In a real-world scenario, Falcon Complete would have stepped in to contain the host and block authentication for the account using the Falcon Identity Threat Protection module. The customer would have then received a remediation notice via email or Message Center, informing them of the steps taken by Falcon Complete to eradicate the threat.

Figure 5. Cerutil.exe used to download malicious DLL VERSION.dll

Figure 6. Notepad++.exe sideloading VERSION.dll to load QuasarRAT

Identifying Potential Lateral Movement

Once command-and-control (C2) connectivity had been established, CrowdStrike observed the threat actor use QuasarRAT malware to conduct keylogging activity, allowing them to attain credentials for the domain administrator account. Falcon Complete quickly obtained a process memory dump of the Notepad++ process utilizing Falcon Real Time Response to validate that the credentials for kizumi.da had been captured by the keylogger and stored locally on the system under the file path C:\Users\kizumi\AppData\Roaming\Logs\2024-03-25-log. Consequently, this keylogger file was observed being exfiltrated back to the threat actor’s C2 infrastructure.

The threat actor then used their valid credentials with domain administrator privileges, conducting additional network reconnaissance to identify high-value targets — including a Domain Controller, which was targeted for further credential theft. In addition, the threat actor abused the Server Message Block (SMB) protocol to transfer files throughout the environment.

Later, the threat actor used tooling matching indicators consistent with ExMatter to search for and gather potentially sensitive information such as Microsoft Office documents and image files from multiple systems. This allowed Falcon Complete to conclude a second adversary, ALPHA SPIDER/BlackCat, was involved. The data was then exfiltrated via SFTP to threat actor-controlled infrastructure.

Figure 7. Exmatter data exfiltration via SFTP to threat actor-controlled domain

As the threat actor’s final act, a ransomware binary was downloaded and executed remotely across all victim systems via a renamed copy of PsExec.exe. This destructive act encrypted files, terminated critical processes and performed actions that would inhibit system recovery.

It should be noted that had this not been a controlled simulation, at any point from the first detection onward Falcon Complete would have enacted countermeasures such as network containment, artifact remediation via Falcon Real Time Response, or the blocking of affected accounts via Falcon Identity Threat Protection in order to stop the breach — all while maintaining close communications with the customer. Our comprehensive response capabilities set Falcon Complete apart from any other managed service.

We Dominated the Evaluation — and CrowdStrike Offers Even More

While these test results are fantastic for CrowdStrike and our customers, they don’t come close to telling the full story of the power of the Falcon platform and CrowdStrike services.

Falcon Complete delivers full-cycle remediation: The “R” in “MDR” was notably absent in this evaluation, which focused solely on detecting and reporting adversary activity and did not extend to response or remediation. Most MDR services stop at identifying and investigating threats, leaving the response to the customer with “guided recommendations.” Falcon Complete goes beyond this by performing surgical remediation for endpoints, identities and cloud workloads, containing hosts, removing malicious artifacts and restoring systems to their normal state.

Falcon Adversary OverWatch unearths unknown, advanced attacks: MITRE didn’t specifically evaluate Falcon Adversary OverWatch, but Falcon Adversary OverWatch threat hunters were involved in the active evaluation and were crucial in identifying and reporting steps to MITRE. As adversaries evolve their tradecraft to evade automated detections, Falcon Adversary OverWatch quickly adapts and adjusts to make sense of early signals and complex attack patterns to uncover previously unknown threats.

Falcon Next-Gen SIEM: CrowdStrike Falcon Next-Gen SIEM integrates key data and threat intelligence, including third-party data, into a single console so analysts can see the full scope of any attack with instant correlation of Falcon and third-party sources. With CrowdStrike’s industry-leading threat intelligence and investigation, Falcon Next-Gen SIEM simplifies and accelerates investigations.

Falcon Prevent was disabled: Prevention capabilities of the Falcon agent were not permitted. This highlights the skill of the Falcon Complete team, which was still able to report 42 of 43 (98%) adversary steps in the detection coverage category and provide recommendations to MITRE in record time. In a real-world scenario, CrowdStrike’s experts, combined with the full capabilities of the Falcon platform, would have disrupted all of the attack attempts before attackers could achieve their objectives, and most of the activity would have been stopped at first contact.

CrowdStrike strongly values participation in independent testing and evaluation. Exercises like the MITRE Engenuity ATT&CK® Evaluations: Managed Services evaluation give customers invaluable, unbiased insight into the performance of cybersecurity solutions. They provide insights into critical features and capabilities and how these will affect the outcome when protecting against real-world adversaries. Participating in these tests also drives innovation, helping CrowdStrike to develop new features that support our mission of stopping breaches.

Additional Resources

• Read the press release.
• See the full results of the MITRE ATT&CK evaluation.
• To register for the CrowdStrike CrowdCast on the MITRE ATT&CK® Evaluation: Managed Services, visit here.
• Learn about another recent MITRE evaluation of the CrowdStrike Falcon platform in this blog: CrowdStrike Achieved 100% Protection, 100% Visibility, 100% Analytic Detection in MITRE Engenuity ATT&CK® Evaluations: Enterprise.
• Read about how CrowdStrike Performed in MITRE ATT&CK Evaluations for Managed Services, Round 1.