I was talking with a CISO the other day who had been dealing with a security incident that seems to become more significant the more he investigates it. He's in the unenviable position of having to not only inform his management that the incident occurred – but having to continually inform them that additional issues are being discovered. It seems as though the incident will not go away.
Dealing with the issue itself is clearly the priority – stopping the damage and doing triage. Once things are under control, a more detailed analysis should take place. Some people call this a "post mortem" – but I've always preferred the term root cause analysis (RCA). The RCA is where you get the answer to that question – "How could this have happened?" It should also tell you "How could this have been avoided?"
A thorough and detailed RCA should be the final product your incident response team produces
for every incident that occurs. It should identify the "who, what, when, where and why" of what happened; and the RCA should start as soon as the situation is under control.
The "what, when and where" questions tend to be part of the triage process, and should be the priority initially – particularly with an incident that is more extensive than it first appeared to be. "Who" may well take additional investigation – and you may never know for certain. The "why" may well be the most important part of the process. Why did it happen? What vulnerability was exploited? Should it have been identified and remediated before the exploit occurred? What controls were lacking? Why wasn't an attack detected and the damage prevented?
Communicating with management is difficult in these times. They will be concerned about impact to the business and the organization's reputation. Your security awareness program should include your senior management team. Make sure you can respond effectively when something bad does happen. Ensure that your incident response team can handle whatever may happen. Have a detailed and effective incident response plan. Test it regularly and train your team constantly.
If you don't have sufficient in-house staff, then budget for additional incident support from vendors that offer those services. Go through scenarios with your incident response team, such as: "What would we do if X happened?" It will make the team much more effective and you'll be amazed at the issues you will uncover. Effective incident management is a core component of our jobs and one we need to excel at.
