Incident Response Fundamentals – Communication

While there are a number of ways to err during the course of responding to an incident, it never seems to fail that the number one misstep I have continued to see comes down to internal communications among those working on the incident and communications to stakeholders. To that point, it’s astonishing that something seemingly as basic as communication can be the downfall of an otherwise great response to any incident. However, if you consider the number of stakeholders normally involved during an incident, let alone the leadership concern and focus that comes along with one, it shouldn’t come as a surprise that communication is perhaps one of the most important, and yet, most overlooked aspects.

Communications Rhythm

First and foremost, you need to establish and follow a communications rhythm. Generally speaking, that should map back to the overall severity of an incident and be agreed upon ahead of time.

As an example, the chart below details an organization with three severity levels and an associated communication rhythm, and intended audience, and method of communication for each.

As also demonstrated by the example chart above, communication during an incident should not be confined to just written communication nor should it end when the situation is under control. Having both conference calls for the team to sync up and leadership to ask questions and provide guidance can be critical; additionally, having a read-out of the incident along with reviews of failures and recommendations for moving forward is an important step at the tail end of an incident.

Cisco Security Incident Response Services Template

For the purpose of this blog post, we’ll focus on daily (or routine) communications only. While some organizations may have systems that assist with the workflow of an incident and have the ability to track and record actions taken, generally leadership is not going to be in those systems or want to log into a system they would use maybe once a year. It’s important to elevate your communication level and consider communication from a leader’s perspective, to include the fact that most leaders are mobile and are keeping up to date via their mobile technology. The template I’ll walk through (see below or download here for your own use) has been used and refined for a number of years now in both Fortune 100 companies as well as with all of our Cisco Security Incident Response Services customers.

 

Key Elements

At the very top of the communication are housekeeping items. These should include when the incident was reported/began, who is working the issue, and perhaps most importantly when the next update can be expected.

In keeping with the concept that leadership “shouldn’t need to scroll”, we want to keep as much of the executive level information at the top as possible to create a one-pager. Having an evergreen Latest Update and Current Impact section at the top will generally provide leadership with the key elements they need to know about to stay informed and make business decisions. Latest Update should be pretty straightforward and give a current state of affairs. Current Impact can be tougher for responders, as I tend to see some responders looking at low-level detail such as number of machines impacted or staying in the weeds for impact. Impact should be directly tied to the business and convey the actual impact the incident is having. For example, is manufacturing down? Are sections of the organization unable to authenticate and login to get their work done?

Areas of Concern should be the location where you communicate any help or assistance needed from leadership. It’s not an area for commentary such as, “we have a malware outbreak”- that should be a given. Perhaps budget is needed to procure a solution to segment the network, or the team is having a difficult time interfacing with a business unit to assist in responding.

As we move toward (typically) the bottom of the first page, we have Action Items. Action Items are perhaps my favorite part of the report as a leader, because it is a mechanism to understand what the team is actually working on, what has been done so far, and when the expectation is they will be completed.

As you can also see by the template, we begin to color code items and leverage formats such as strike-through font. I’m a fan of leaving the items that have been completed, crossed off, as it shows progress. Additionally, throughout the template, we use the color green to indicate new information from the previous update. Again, given that time is always the most precious resource anyone has, allowing recipients the ability to quickly focus their attention on the green sections and read only new information, can save up to 80% of their time versus reading each update in its entirety.

While the Intelligence Summary is a little lower in the order of the template, the template is designed to be modular in nature and can flex depending on the issue at hand. For example, the intelligence pertaining to a Qakbot outbreak may not be that relevant or exciting to leadership, but a nation state adversary who has just exfiltrated the company’s intellectual property should be moved to the first page, as it may influence leadership to make different business decisions.

Finally, but again critically important, is the Recommendations section. We consider this a parking lot for things that are discovered and discussed along the way, but perhaps do not have time to focus on given the other priorities. Per the above commentary around a final debrief following an incident, the Recommendations section is usually the top item discussed in the debrief and rather than try and recall what happened a week or two ago, documenting these along the way is invaluable.

Conveying Doubt

Incidents are always fluid situations, and as such it can be very difficult to definitely state findings with 100% confidence. As such, when communicating throughout the incident, we use phrases and terms that allow us to provide a certain level of speculation based on our years of experience and knowledge. We provide this as an appendix, so that readers can reference it along the way and feel more confident in the findings being presented along the way. We have recently synced this language with our Talos brethren, which they recently blogged about here: http://blog.talosintelligence.com/2017/08/on-conveying-doubt.html

Final Thoughts

Clear, crisp, consistent, and constant communication is critical and calming. Leadership tends to get antsy when there are issues ongoing and they aren’t sure of the current status. Assigning an Incident Coordinator/Commander to take point on communications and running meetings will help to ensure they know what is going on, and will empower them to know when they can next expect an update, what the current situation is, and what has been done. This will then allow leadership to worry about other matters and perhaps more importantly, allow the response team the ability to focus on the incident at hand.

As every organization is different, feel free to use the template and modify it as you see fit. I’d also love to hear back about other areas you think are critical for these routine communications, so please drop a note if you have something to add.