In an earlier blog post, I touched on the history of MITRE ATT&CK™ coverage and how at Tidal we think of coverage in terms of the confidence a user has in their defenses to combat the threats that are most relevant to them. Understanding the capabilities a user’s tools provide is crucial to helping that user understand their confidence, and what they can do to improve it.
I have spent most of my working life working closely with vendors to map their capabilities to ATT&CK, integrate ATT&CK into their products, and test their solutions against ATT&CK. Almost every engagement has been centered around enabling vendors to give customers the best solutions possible. Even as ATT&CK Evaluations became more of a requirement for vendors to participate in than a willing exercise, for the vast majority, they still understood the intent: we need to see how we are doing, and how we can improve.
I believe this goes to the general sentiment of the ATT&CK community: be threat-informed and know you can always improve. With this positive sentiment has come a positive trend. More and more vendors map their solutions to ATT&CK. They want to make sure that their solutions address the known threat, not just the hypothetical, and they use ATT&CK to communicate their value proposition to their users. It is no longer about only becoming that solution that can show breadth of ATT&CK coverage, but embracing niches addressing specific techniques in ATT&CK.
However, despite best intentions, and despite its widespread adoption, little progress has been made towards allowing users to know what their coverage is.
Why is this?
At the core, it is because ATT&CK was developed as an abstraction of adversary behaviors, a knowledge base that could serve as a unifying language for different stakeholders with different perspectives and skillsets. With that abstraction comes a lack of specificity around the exactness of behaviors (e.g., procedural information), and this lack of specificity drives a disconnect between reality and the conceptual level. That’s right: the thing that makes ATT&CK one of the most powerful concepts in cybersecurity and so broadly supported, is also the thing that makes its broad adoption challenging.
Take the experience we all see with every release of MITRE Engenuity’s ATT&CK Evaluations. I could write a book on all the misuses and misconceptions I have seen around this effort, but let’s focus on the thing that’s most criticized: vendor claims of high levels of coverage. The culprit is simple to identify. Users want ATT&CK Evals to tell them what tools defend best against the threat. Vendors want to be able to tell those users they are best at defending against the threat.
From this brings two main issues:
- Configuration confusion...
- ...which leads to over-generalization
Configuration confusion comes into play as some vendors come in with self-proclaimed default configurations, some with custom rulesets, and some come in not being completely transparent. You hear a lot of “but how would this work in the real world” type of comments every release, whether they’re fair or not. Frankly, in the end, vendors are in a no-win situation. They can come in with a default config and no one believes them, or they can come in with some custom ruleset and be told that solution isn’t deployable.
The result of over-generalization is vendors pointing to how well they did at a level that makes it look better than they are. Look at the last round of the Enterprise Evaluations and you will see marketing claiming coverage of all tactics, techniques, and steps (similar to procedures). But even if we look at steps, that doesn’t imply full ATT&CK coverage. Sure, ATT&CK Evals (as you could tell by the name) maps to ATT&CK, but they are conducting their emulation at the procedural level. In fact, whereas ATT&CK Evaluations mapped to ATT&CK Techniques in prior years’ emulation plans, this year they didn’t map techniques into their emulation plans. You can still get to the technique level information in the results. The fact the actual test is procedure-focused is a chief reason why ATT&CK Evals tells us “ATT&CK Evaluations are a starting point,” and “there are no winners.”
So, what’s the solution? As I noted above, I believe that most vendors want to be threat-informed and provide the best solution they can to their user base. They want to secure their customers. After all, secure customers mean happy customers, and happy customers mean continued customers. What we need to do is enable vendors to talk about their capabilities in more detail. Allow the vendors to show what you can get out of the box, but then also show how if a specific technique is important to you, how you can defend against that with some special rule or integration. We need to think in terms of, “I have this technique covered in these ways,” rather than the simpler, “I have a technique covered.” It isn’t a yes/no; it’s an evolving gradient based on the evolving threat and evolving solutions. This goes for the vendor who is providing capabilities just as much as the end-user.
The Tidal Product Registry is an example of how vendors are trying to push on all the good things ATT&CK did for the community and move it to the next level. We are working with a wide variety of vendors, some well-established, some startups, some broadly focused, and some focused on specific techniques. But each and every vendor on the platform is united in showing how their solution address threats and doing it in the public eye with their name behind it. Why? Because it enables users and prospective users to be empowered through their transparency. Users can know what a vendor has and where. Users can explore options to know what they could do differently to improve. The vendors have decided that being transparent about their capabilities as a means of empowering users is more important than making broad claims about their capabilities.
SentinelOne is the latest vendor to join the Product Registry and is available today within the platform! They made their data available in the Tidal Community Edition, which everyone can freely access. They join the likes of recently added Cybereason and launch vendors Atomic Red Team, AttackIQ, BreachBits, Olaf Hartong’s Sysmon Modular, Picus, Remediant, Scythe, and Trinity Cyber. We continue to work with a variety of vendors to structure their data and provide the extra context we feel is necessary to accurately reflect ATT&CK coverage. If you want your solution provider in the Registry, let them know you think this is an important effort to support. If you are a vendor and would like to join the Registry, please reach out. We are here to help you through the process, and have the experience needed to ensure your capabilities are accurately reflected in the Registry.
