This blog is by YETI Holdings Inc. (NYSE: YETI) Senior Security Architect Eric Ooi and was originally published on his personal site. The process he outlines in this blog was used to evaluate a variety of combined endpoint protection platform (EPP) and endpoint detection and response (EDR) solutions, and ultimately led him to choose the CrowdStrike Falcon® platform to protect YETI endpoints. The views expressed in this article are the author’s alone.
Like most cybersecurity professionals, you’re looking for an EPP that protects against current and evolving threats, is easy to deploy and manage, and is ultimately invisible to end users. Today, there are dozens of these platforms available, and choosing the right one for your business is a daunting task. With each vendor claiming their solution is “next-gen-everything,” is the highest rated, and is easiest to use, how do you select what’s best?
Last year, I conducted a bake-off among three endpoint security solutions. My goal was to replace an incumbent legacy system with a modern endpoint protection platform (EPP) and endpoint detection and response (EDR) solution. I researched online for methodologies and guides to evaluate and compare solutions but didn’t find much. The SANS Advisory Board mailing list had some good tips and together with my own curiosity and experience, I created the following walkthrough. The entire experience was one of the most fun I’ve had and I hope my guide helps you in your own evaluation process.
First and foremost, determine your requirements. Some questions to ask yourself:
- What operating systems do I need to cover?
- Am I worried about fileless malware and PowerShell-based attacks?
- Is endpoint detection and response (EDR) capability important to me?
- How will this new solution integrate and enhance my current solutions?
- Do I need a solution that provides managed services such as monitoring or incident response?
Based on these questions, create a “capabilities checklist.” This will be your specific criteria for a valuable experience. A sample is below.
|Single agent or multiple products?|
|Search for IOCs: MD5/SHA256 hash, registry keys, filenames|
|View downloaded files, DNS cache, network connections|
|Live forensics: quarantine system, view/kill processes, download files for analysis, view active network connections|
|Integration with existing solutions: Firewall, SIEM, etc.|
|Run on-demand manual scans on files/directories|
|Process for handling false positives|
|Create groups of systems and apply specific policies per group/system|
|Disable an agent from the central console|
|Low resource usage|
|Preventing/detecting malicious PowerShell scripts, fileless attacks, suspicious command usage|
|Detect and prevent malware|
|Unique features of the platform|
There are plenty of guides, reviews, and reading materials available to get an idea of what solutions will meet your requirements, at least on paper. Good starting points are:
- MITRE ATT&CK Framework
- Security publications
- SANS Reading Room
- Your security peers
Depending on your resources and timeline, try to narrow your list to 2-4 solutions you feel meet your minimum requirements.
Reach out to the vendors you intend to evaluate. Prepare a good list of questions and have them walk through your use cases. Typically, these pre-sales calls will involve an account executive and a sales engineer that will introduce you to their solutions, speak to their differentiators, and conduct demos. You may even be able to weed out some of the vendors just in this initial stage.
Throughout the process:
- Don’t reveal which solution you’re leaning towards.
- Document your findings.
- Ask lots of questions.
- Be courteous.
Truth be told, no solution is perfect, but leveraging a standard testing methodology will enable you to objectively evaluate each solution in a fair and repeatable manner. You can use the following methodology to complete your “capabilities checklist” from the Determine Requirements phase.
- Setup test “victim” virtual machines, completely separated from your production networks, running each of the operating systems you’ll need to protect. Virtual machines will allow you to quickly revert to a clean state. Cloud or container-based solutions work well for this.
- Setup an “attacker” machine using a Kali virtual machine.
- Download a collection of known malware samples (e.g. Petya). Lenny Zeltser has a great list of free malware sample sources.
- Set up a malicious PowerShell attack using Unicorn (https://github.com/trustedsec/unicorn).
- Run the known malware samples and observe which are detected and prevented. Test this both on and off network to accurately evaluate any cloud or offline prevention capabilities.
- Observe resource usage using the respective operating system’s process monitor. Note resource usage during idle and malicious activity.
- Run the Unicorn PowerShell attack using Kali. Try this in both “full prevention” mode and “detect-only” mode to test prevention and EDR capabilities.
- Run additional types of attacks using Kali: https://www.offensive-security.com/metasploit-unleashed/
- Run suspicious commands like netcat or dump password hashes and note if the activity is prevented or detected. Observe whether or not the solution brings these to your attention or if you have to dig to find what you’re looking for.
- Test the ease of deployment and uninstallation. Both will be equally important to any teams that will be supporting the management and maintenance of the platform.
- Test “bypass,” “detect-only,” or “disable agent” modes in the event you’re asked to disable the protections for troubleshooting purposes.
- Move a system between policies and observe how long it takes for changes to apply.
- Test remote forensic capabilities. View running processes, download files, kill processes and view netstat information.
- Observe how easy it is to search for IOCs: hashes, filenames, IP addresses, hostnames.
- Create whitelists/blacklists for specific files and test to see they are actually allowed or blocked.
- Test network containment of an endpoint. Is the system truly isolated and unable to connect to anything other than the EPP console?
- Get a feel for the UI. Are the features you care about the most easily accessible and intuitive to use? Is it slow or difficult to navigate?
- Test any unique features of the platform. Do they run as well as the vendor claimed? Do they add value to your workflow?
- Try to seriously break it. Click on everything in the console. Assuming you’re only running against test virtual machines, this shouldn’t break anything. If you run into issues, reach out to the vendor’s support team and evaluate how responsive they are. Better to know this now than when you’re in a real emergency.
- With your completed “capabilities checklist,” review your findings and observations. If there are any follow up questions, get these answered by the vendors in writing.
- It’s not an exact science, but the results from your checklist combined with your experience working with the pre-sales and support teams should give you a good idea of which solution is right for you.
- Get quotes for all the solutions you’re evaluating. Even if you’ve decided on which solution you want to go with, use the other quotes as a way to drive down the price for the solution you want. Let each vendor know you’re looking at competing solutions and ask them to include any training, conference tickets, or additional incentives to create the most compelling offer. Don’t feel bad about this, it’s their job to sell to you.
- Depending on your timeline, you may not have a choice for when to make the purchase, but typically you’ll want to go for the vendor’s year-end or quarter-end to get the best prices and incentives. Obtain in writing what the expected renewal process and prices are.
Choosing the right endpoint protection platform solution is critical to protecting your business from today’s ever-evolving threats. With an increasing number of EPP solutions to choose from, cutting through the marketing noise is a significant challenge. By applying a standardized evaluation and testing methodology, you can ultimately make an informed and objective decision on the right solution for your business.
Visit my website and learn more: ericooi.com
Additional Resources from CrowdStrike
- Learn why organizations choose CrowdStrike by visiting the “Why CrowdStrike?” webpage.
- Get information to help you navigate a crowded vendor landscape, download the “CrowdStrike Endpoint Protection Buyers Guide.”
- Find out what third-party analysts and evaluators have to say about the Falcon platform by visiting the CrowdStrike Industry Validation webpage.
- Download the white paper: Endpoint Detection and Response (EDR): Automatic protection against advanced threats.
- Learn more about the MITRE framework in the white paper: “Faster Response with CrowdStrike and MITRE ATT&CKTM.”
- See what third-party security testers and reviewers are saying about Falcon.
- Test CrowdStrike next-gen AV for yourself: Start your free trial of Falcon Prevent™ today.