Finding value with a security automation orchestration and response (SOAR) product can be challenging without direction. If your organization doesn’t have the internal expertise to provide that direction, you could easily find yourself adrift and unable to realize the benefits promised by this new capability. A majority of organizations fall into this category which has led to many unsuccessful deployments across the industry, leading to frustration with the tooling, and frustration with vendors.
These organizations need a partner to ensure their success. With this goal in mind, I started digging into the Cortex XSOAR customer playbook telemetry to figure out what success looks like and build an understanding of what a customer journey looks like for customers that are finding success.
Understanding What A Successful Customer Journey Looks Like
The XSOAR customer telemetry is something customers have opted-in for and comprises data from about 600 customers across 22 industries and 28,000 playbooks. Before digging in to properly understand what a successful customer journey looks like, I had to perform three critical steps which I feel gave me the highest fidelity understanding of what customers are doing given the data:
- Create a timeline to understand the stages of deployment. I constructed a timeline for each customer starting with their first playbook execution, and then analyzing playbook creation in the first thirty days, ninety days, and six months.
- Develop a natural language processing engine to normalize playbook names. Customers tend to name their playbooks whatever they want. This makes it difficult to do a frequency analysis on playbook usage because that data set will be heavily biased by out-of-the-box playbooks due to a lack of uniformity in naming. I solved this problem through topic modeling the playbook names.
- Differentiate between direct and indirect execution of playbooks. With the XSOAR product, playbooks may be executed as part of a workflow, or they may be called from another playbook. Differentiating between direct and indirect execution allows us to understand high-level customer objectives instead of being biased toward out-of-the-box, enrichment type helper playbooks that tend to be called with very high frequency because they are helpful for most customer objectives.
Since our data starts with the first playbook executed, this would be the proof of concept (PoC) stage for many customers, so let’s start there.
The First Thirty Days: From PoC to Functional Proficiency
The first thing new customers need to become familiar with is creating playbooks. There is no better place to start than tackling high-impact process automation objectives that are low-impact to the overall environment. Generally, customers will start by optimizing their phishing workflows and building out their alert and incident management pipeline. A great first step is to start building your alert management pipeline by integrating the “Default” playbook, which operates as a fall-through condition and can be used to do nothing more than enrich alerts.
A helpful resource that XSOAR provides are content packs that deliver prebuilt playbooks that will form the building blocks for your top-level objectives, reducing development time and directly improving time to value. By far, the most widely adopted content pack is the “CommonPlaybooks” pack which is used by over 80% of our customers and contains playbooks for enriching information for all your common indicator types as well as the ability to generate criticality scoring.
Ninety Days In: Navigating Early Adoption
We see customers transitioning from being a prospect to a customer within the first 90 days. At this stage, the training wheels are coming off and it’s critical to build robust development practices around playbook creation. You would never roll code to production without testing it, and you really shouldn’t do this with your SOAR implementation either. A common pattern we see is people naming playbooks with the words “dev” or “test” which is a clear indication of this pattern.
From a development perspective, customers are continuing to build out their alert management and phishing use cases, but we’re also seeing integration from threat intel sources as well as user management. User management use cases are particularly interesting, especially around the automated disabling of corporate accounts in response to changes in employment status. This creates assurance that user accounts are disabled. A major benefit of the orchestration capabilities available through SOAR is the creation of auditable processes, which are guaranteed to make the compliance team smile.
Six Months In: Becoming An Automation Leader
A key indicator that a deployment is going successfully is when we see customers starting to transition from managing their alert pipeline to developing use cases that are going to be generating alerts themselves, such as automated threat hunting in Splunk. To get to this point, customers must have gained experience creating playbooks and their robust processes around developing and testing playbooks, building confidence to start working with their crown jewels.
Imagine what use cases you would start to fulfill if your SOC analysts weren’t spending all their time chasing alerts… this is the panacea that vendors are selling you when they talk about SOAR. It’s not an easy journey for everyone, and depending on the resources available, this six-month roadmap could take a year. The important thing is that you have a roadmap and can build expectations based on it. If you’d like to learn more about this research with a deeper dive into the use cases and metrics for tracking your progress, register for Cortex Symphony 2023 here.