Succeeding By Planning to Fail

by Martin Fisher

As security professionals, it’s hard to admit to to our bosses (and ourselves) that all of the work we’ve done to prevent compromise sometimes isn’t enough. We don’t like to think about the possibility that the money and time invested in technology might not prevent an incident from occurring. That’s why I proposed, in my previous article, the following basic truth for Incident Response Leadership:

Basic Truth #1: Assume You Will Fail

One of the issues we face in Incident Response is how we frame success and failure. Too often we define our success with phrases like, “we’ve never been hacked” or, “our systems have never been breached”. These phrases fly in the face of the fact that no system is 100% secure. They dismiss the fact that a sufficiently motivated (or lucky) intruder can get in.

So, re-framing and redefining “success” is key to actually being successful. How do we do that?

First, we have to publicly acknowledge to our bosses, peers, and team that we expect that some small percentage of hosts and devices on the network will someday become compromised. It could be malware, it could be an intrusion; it could be almost anything. We need to help our teams and bosses realize that it’s not only okay to find these flaws, but that it’s actually a vital part of keeping our environment secure.

Second, we have to have a set of plans, procedures, and technology in place that allow for continuous monitoring and detection of problems in the environment. As leaders, we need to push for thorough and repeated examination of our environments and celebrate each and every compromised system our teams identify, contain, and eradicate. We must inculcate a philosophy that finding “nothing wrong” is more a sign that detection systems and processes need improvement, than it is a sign of successful prevention.

Lastly, and most importantly, we have to build the right networks of people, processes, and capabilities to make the most of the monitoring and planning. As Incident Response Leaders, our most critical mission is to build effective individuals and teams that can stand up to the pressures of Incident Response.

But, you ask, how do I do this? It isn’t easy – but Incident Response Leadership rarely is…

To start the process, you need to sit down and honestly assess your network. Bring in some trusted outside advisers if you need to. Are you really keeping anti-virus updated on all of your systems? Are you deploying operating system and application patches in a timely fashion? Are your IDS/IPS systems workable? How much screening do your firewalls really do? If you put on your blackhat – how many ways could you penetrate your network?

Once you’ve completed the process of seeing exactly how secure (or insecure) your environment really is, take a deep breath. The natural response to this kind of in-depth assessment is to think that the world is collapsing and that only huge amounts of effort can ever fix it. Remember, you aren’t here (necessarily) to fix those infrastructure issues right now; you are here to develop the ability to respond to incidents right now.

Now, take the list of perceived weaknesses and map out, using existing resources, how you intend to respond to this kind of incident. Don’t develop detailed plans right now – that comes later. Just identify how you can respond with what you’ve already got. A quick spreadsheet should do the trick here.

Next, invite your boss to have a cup of coffee with you. Let the boss know what you’ve been doing and the relative assessment of the network (remembering that the sky, more than likely, isn’t really falling). Show the boss how you intend to respond to the potential incidents using your map. The key to this meeting is being calm, professional, and not sounding like a) Chicken Little or b) you are about to ask for a ton of new resources. You need to show how you are going to realign your existing resources (which have been good enough so far, right?) to meet the challenge.

The key part of that conversation is to start the process of setting realistic expectations with the boss. Share the truth that you’re doing everything you can; that a lucky and/or motivated adversary could still compromise the system; and that, being the Incident Response Leader that you are, you are going to develop the plan and the team to identify, contain, and eradicate any and all intrusions.

Once you’ve got buy-in from your boss you’re ready to tackle the next Basic Truth: Have a Workable Plan. But that’s for the next article.