April 30, 2020
Recently MITRE conducted an autonomous evaluation of 21 cybersecurity products to support the industry and government make well informed conclusions to battle an ever-evolving cyber security threat. The outcome of the evaluation has been released to the general public.
Unlike traditional evaluations wherein, scores are assigned to solution capabilities, this evaluation laid emphasis on illustrating how detections occur. For the same, the test criteria were defined on the lines of replicating the techniques used by known threat actor groups – APT3, and APT29 being the latest.
The team chose to emulate the techniques from above listed threat actor groups against the cyber security solutions as they use sophisticated real life modus operandi and implementation methods via custom malware and execution methods.
Regarding the detections pertaining to the evaluations, detections have been categorized according to the identified unique techniques. As with any solutions, the cyber security solutions have multiple detection capabilities ranging from specific behavior based detection, Indicators of Compromise, collection of minimally processed data patterns, etc. MITRE has considered 9 latent detection categories.
The ATT&CK Evaluations portal illustrate a visual dashboard, which facilitates the users to select specific vendors and compare beside each other of how they identify and tackle the techniques. Along with the evaluation results, the team also released a DIY APT29 evaluation leveraging CALDERA, which is a system created by the MITRE using the existing knowledge base. This provisions everyone to assess the cyber security solutions in their respective environments against the same threat actors.
From an overarching standpoint, this evaluation may have certain limitations:
- As emulators primarily focus on publically available threat reporting, not all the malicious activities faced by the organizations may be covered in the available public reporting.
- Although defined categories and outcomes, consistently cover all the evaluated security solutions, there may be certain areas subject to discretion, based on respective environments.
It is highly recommended to review this report and we believe, it should be as useful as looking through the eyes of potential cyber criminals. That being said, the defense in depth capabilities and zero trust principles which we recommend, always help to circumvent sophisticated attack vectors throughout its lifecycle.
For more information, check out MITRE press release
