This week marks the release of results from the second iteration of MITRE’s recurring ATT&CK Evaluation program, and I could not be prouder of how CrowdStrike Falcon® performed. 

MITRE’s results have once again validated the mission and focus of the CrowdStrike® Falcon platform, which has been proven time and time again to stop breaches by:

1. Providing excellent protection, by design
2. Eliminating noise and alert fatigue
3. Equipping your analysts with the right data, context and guidance to operate at speed and stay ahead of the adversary

Head-to-Head Against the Adversary

CrowdStrike’s 2020 Global Threat Report showed that more than half of intrusions observed in 2019 didn’t start with malware  — the attackers got their initial foothold via fileless techniques. The attackers then proceeded to slowly and methodically work their way through the organization toward their ultimate objective. A modern approach to preventing breaches demands endpoint protection solutions that are effective beyond malware. 

The MITRE ATT&CK® Evaluation represents the industry’s most in-depth investigation into how endpoint protection solutions behave when subjected to complex, real-world threats. An ATT&CK  Evaluation pulls from real-world observations on how a known adversary has operated in the past (in this case, COZY BEAR, aka APT29) and then crafts a repeatable test plan designed to emulate this adversary’s behavior.

In this year’s APT29 emulation, MITRE executed a series of attacks spanning the full ATT&CK spectrum covering 134 separate test procedures. The simulated attack stages ranged from initial compromise via a weaponized Microsoft Word document to exfiltration of encrypted data. MITRE subjected 21 endpoint detection and response (EDR) vendors to this same battery of tests — with all prevention capabilities disabled — and tracked their ability to surface suspicious and malicious activity for a security analyst.

EDR That Empowers — Not Overwhelms

Today’s Security Operations Center (SOC) does not need more data. Security analysts desperately need help making sense of the data they have in time to stop breaches. The newspapers are full of stories of intrusions and breaches where the signs of attack were present for weeks or months before hitting the security team’s radar, because the security teams were overwhelmed with weak non-correlated signals. Three problems are at the core of this issue:

• Too much data
• Poor prioritization
• Difficulty tuning solutions to perform as expected

CrowdStrike is on a mission to stop breaches, and we have innovated tirelessly to solve these challenging problems, in strong partnership with our customers all over the world. MITRE’s test results provide strong validation that together we are winning this fight, making strong inroads against sophisticated threats. Let’s examine the keys to this success.

1. Transition From Alerts to Incidents 

CrowdStrike has changed the game in 2020 with our new CrowdScoreTM technology, providing defenders with rich security incidents — replacing cumbersome security alerts that are difficult to understand and act on.  

In MITRE’s testing, CrowdStrike delivered world-class levels of detection providing visibility into every stage of the simulated attack executed by MITRE, solving the problem of “too much data” by putting it all in its proper place for the analyst. No other EDR solution was able to demonstrate the ability to tie together so much critical information into such a small and powerful package.  CrowdStrike was by far the class leader in providing correlated telemetry because we leverage our proprietary CrowdStrike Threat Graph® technology, which never loses the context of an event. 

Alerts are traditionally the kickoff point for the real work that happens in the SOC. Stopping breaches means that investigators need to understand and respond to threats in minutes — within the breakout time window. To get this information, analysts can’t rely on EDR solutions that require them to hunt for the supporting data and the context they need to make sense of a threat and plan action. Such actions take a great deal of time and expertise, and both are in short supply during a real attack. 

Another limiting design factor in being able to reduce alert fatigue, prevalent in common EDR solutions, is the “dumb sensor” model, which is an endpoint that does little processing and simply collects and sends data to another system for analysis. The design choice to send data to another system rather than processing it on the endpoint itself is a huge missed opportunity. It results in higher alert quantities being sent for processing, causing unnecessary network traffic. CrowdStrike Falcon®’s lightweight single agent changes the whole equation, making data processing and decision-making on the endpoint not only possible, but preferable. This brings not only highly accurate, real-time prevention and detection, but also intelligent filtering that determines which data should be made available to the analyst and under what conditions, a design which is essential to assist with incident prioritization and the reduction in alert fatigue.

Incidents combine alerts with the rich correlated telemetry from CrowdStrike’s proprietary Threat Graph, allowing defenders to see the full attack narrative in real time as can be seen below.  Because our cloud was purpose-built for real-time security processing, one of many strengths is the speed with which alerts are analyzed and converted into incidents in Threat Graph. The temporal nature of Threat Graph allows multiple detection methods and algorithms to run against the data simultaneously, leading to near-instant results as the data stored in Threat Graph becomes immediately available for viewing, visualization and retrospective searches.  By minimizing the number of alerts a security team needs to deal with and maximizing the context of each individual incident, we give security teams the information they need to meet the 1-10-60 challenge.

2. Smart Prioritization and the Elimination of Alert Fatigue

Empowering analysts to work with speed and efficiency is step one in winning the fight. Step two is ensuring that analysts are always working on the most important threats first. 

CrowdStrike CrowdScore^TM technology was developed not only to consolidate workloads but also to help organizations quickly understand their current threat levels and to intelligently prioritize the critical threats in real time. CrowdScore eliminates alert fatigue and ensures important events are not falling through the cracks, which could result in breaches even when all of the information is available to detect them.

 

With the full context of an incident, derived from the CrowdStrike Threat Graph, CrowdScore’s smart prioritization streamlines the triage process and presents the most important incidents to analysts via a central incident dashboard, ensuring responders are always directed to the most critical threats first.

When you remember that this test was performed in a pristine lab environment and includes no benign activity, no “noise,” and there were no considerations for false positives, you’d expect a relatively low number of alerts. The era of the original SIEM showed us that generating dozens upon dozens of alerts for a single detection is not a sustainable model for organizations that see hundreds of attacks each day. 

Successful security teams are the ones that can correctly select the critical incidents to address and when to address them, which requires correlated alerts and incidents as can be seen through the screenshots from the evaluation. Low volumes of alerts combined with intelligent prioritization is key to success.

3. It Works Out of the Box With the Lowest Total Cost of Ownership

We often hear that Falcon is the easiest product to install and operationalize.  Why?  Because it works right out of the box. This is key to successfully fighting the adversary — ensuring your EDR solution does what you expect, especially in the heat of battle. This is the core philosophy that drives CrowdStrike: Do what is needed to protect customers against breaches without overwhelming your security analysts.

Working out of the box, without requiring months of expensive and labor-intensive integration, also impacts total cost of ownership (TCO), something CIOs care about deeply. With its high detection rates, low false positives and very little configuration and tuning, Falcon delivers on driving down TCO (see for yourself in this  Forrester Total Economic Impact Report). 

In this evaluation, the CrowdStrike Falcon® platform once again demonstrated our ability to deliver truly top-notch results with our standard, out-of-the-box product. No special-purpose content was built for the evaluation, and no in-test configuration changes were used to surface data that a real-world customer installation would miss. 

There is a massive difference between a product that works out of the box and one that only works when the vendor is actively propping it up with the full power of its engineering and support organizations. The results that MITRE published for CrowdStrike Falcon® are the same outcomes that our customers see every day — exceptional detection and response capabilities with extremely low false-positive rates that can be used by organizations of all sizes, without requiring additional people and without the enormous administrative overhead associated with other EDR solutions. After all, the adversary is not going to wait while you ask your vendor to reconfigure the system.

The Full Picture

The MITRE ATT&CK Evaluation focuses entirely on the detection of emulated attack techniques — it is not an evaluation designed to assess a solution’s ability to prevent threats. According to MITRE, emulating the adversary generally does not use the actual adversary tools; instead, they attempt to emulate the techniques as closely as possible using publicly available tools. Emulators rely on publicly available threat reporting and not all adversary activity is covered in public reporting, so emulations will only cover a portion of all adversary activity. It is important to acknowledge the complexity of these styles of tests and that no evaluation can capture the full capabilities of any endpoint protection solution.  

The Falcon platform delivers a wide range of prevention techniques via CrowdStrike’s single lightweight agent, from machine-learning-based anti-malware to behavioral prevention for credential dumping and other attack techniques. As legions of CrowdStrike customers around the world know from experience, a standard Falcon configuration would have stopped COZY BEAR in its tracks at all stages of the attack.

Adversaries don’t stand still and neither do we. CrowdStrike’s sophisticated detection techniques continue to evolve, and we continue to innovate with new capabilities delivered to identify and stop new and emerging tools, techniques and procedures (TTPs). CrowdStrike engineering continuously evaluates the threat landscape, data from our threat researchers and the results of third-party testing — including MITRE’s. The results of this evaluation come on the heels of CrowdStrike being positioned as a leader in The Forrester Wave™: Enterprise Detection And Response, Q1 2020, where we also received the highest possible scores in Product Vision, Performance, Commercial Model, Endpoint Telemetry, Security Analytics, Threat Hunting, ATT&CK™ Mapping and Collection. 

Transparent Testing Matters

CrowdStrike participates in a wide range of independent third-party tests with organizations such as AV-Comparatives, SE Labs, MRG Effitas and others. These testing organizations have become the standard yardstick for measuring the effectiveness of endpoint security solutions, and they work largely by tracking how good the solutions are at preventing malware. While preventing malware remains an important metric, it is not the only one. 

CrowdStrike remains steadfast in our commitment to independent, open third-party testing. Unbiased testing provides our industry with trustworthy insights into the value delivered by security solutions. Testing also helps solution providers such as CrowdStrike to clearly gauge our own effectiveness and drive continuous improvements over time. 

Over the coming weeks, there will be more written by analysts and vendors interpreting MITRE’s testing. In particular, managed services took on an interesting role in this round of testing, and we look forward to exploring these topics in future blogs.

Additional Resources

• Review the MITRE ATT&CK Evaluation Results.
• Download your copy of the 2020 CrowdStrike Global Threat Report.
• Read blogs on the 2018 MITRE ATT&CK Evaluation Results and the 2017 MITRE ATT&CK Evaluation Results.
• Learn more about the CrowdStrike Falcon® platform by visiting the product webpage.
• Get a full-featured free trial of CrowdStrike Falcon® Prevent™ and learn how true next-gen AV performs against today’s most sophisticated threats.