Write the plan before you need the plan

In a real incident, the thing that fails first is knowing who does what.

The first hour of a security incident is loud and confusing, and it's rarely the malware that makes it that way. It's the people. Who's in charge. Who's allowed to shut things off. Who calls the insurer, the lawyer, the clients. Who talks, and who stays quiet. Figuring that out live, while something is actively going wrong, is how a contained problem becomes a crisis.

I ran 24/7 incident response for the Department of Defense before I went into technology integration, and the pattern didn't change. The teams that came through it well weren't the ones with the fanciest tools. They were the ones who had already decided who does what, so the first hour went to fixing the problem instead of inventing the org chart.

An incident response plan is just that decision, written down before the pressure. It doesn't have to be thick. It has to answer a short list of questions while everyone is calm: who has the authority to pull a system offline without waiting for a sign-off, how people reach each other when the usual email and chat might be compromised, who the outside calls go to and in what order, and what gets preserved before anyone starts cleaning up, since the evidence tends to be the first casualty of a well-meaning fix.

The part people skip is the rehearsal. A plan read once and filed is barely better than no plan, because nobody remembers it under stress. Twenty minutes around a table, walking through a realistic scenario out loud, finds the gaps while they're cheap: the contact who left the company, the backup only one person knows how to restore, the step that assumed someone would be reachable at midnight.

You don't rise to the occasion in an incident. You fall back on what you already decided. Decide it on a quiet afternoon, not at 2 a.m.

← All field notes