Despite the complexity of skills required to properly address information security, candidates are rarely put to the test before hiring – opening companies up to a range of risks, from simple workforce churn to breaches.
The problem, say community recruiters and trainers, is not simple failure to bother: effective evaluation of cyber candidates often takes a mix of technology and expertise that few employers have at their disposal, not to mention a skills gap already makes qualified candidates difficult to find. Nonetheless, companies are taking significant chances.
“The risk is obvious,” said Wayne Pruitt, cyber range technical trainer at cyber training platform provider Cyberbit. “In the best case: mis-hires, increased turnover, resulting in increased hiring and rehiring costs. And in the worst case: the issues will come into effect during an incident, resulting in successful breaches. Talented cybersecurity professionals are so difficult to hire nowadays and we have to be certain that we’ve hired right.”
Easier said than done
According to new survey data from Cyberbit, roughly 78 percent of polled security professionals said they were hired for their jobs after just an interview or conversation, without being subjected to further assessment in the form of a quiz, cyber range simulation or similar exercise. Approximately 63 percent of respondents said they believe fewer than half of today’s cybersecurity applicants are actually qualified for the positions for which they are seeking.
The survey’s current sample size is on the smaller side, with about 65 participants so far. However, additional responses are forthcoming and the full results are not scheduled to be published until Jan. 27. Regardless, SC Media confirmed with multiple cyber recruitment and training experts that many companies do not ask cyber job applicants to perform any tests or simulations as part of the interview and hiring process.
“It’s very challenging to create a hands-on assessment and simulation environment for hiring. It requires access to a live network, access to commercial security tools, and preferably a live attack to be running, so candidates can be tested in a real-world scenario,” explained Pruitt.
Most candidate assessments are also done remotely, which means candidates would need to access the test environment remotely. And vice versa, companies would require a means to evaluate the candidate during this exercise. In Pruitt's words, 'this combination is something that simply does not exist.”
What's more, the human resources department members who are responsible for the actual hiring may not be equipped with the knowledge or tools to perform test-based evaluations of prospective talent.
“Technical challenges and exercises can be difficult to maintain and execute for HR professionals that may lack any experience with cybersecurity," said Frank Downs, senior director of cybersecurity advisory and assessment solutions, at the professional IT governance organization ISACA. As such, many of the hiring experiences that I have had have not included technical elements or simulations.”
Jeff Combs, principal at cyber recruitment firm J. Combs Search Advisors, said the main reason such exercises are rarely applied to the broader cyber professional community is unfamiliar with available assessment offerings and their value, plus a “lack of budgetary support.”
These evaluations do happen, but usually "only for highly technical roles” that involve a "development- or coding-intensive discipline," he added. "Penetration testing, software security, IAM [and] cloud security are real-world examples where a technical assessment was part of the interview process.”
And Mark Aiello, president of cybersecurity talent recruitment firm CyberSN, said that job applicants seeking positions as pen testers and red team members are often asked to partake in “capture the flag” and tabletop-type drills. But beyond these exceptions, “I think it is generally not done because it is time consuming, difficult to devise and implement, and administered by too many different people who hold different biases and opinions."
What works
Let's say an organization wants to better assess a potential hire's attributes. What makes for a good job skills test, anyway?
Combs believes it should measure a candidate’s “knowledge baseline, ability to conceptualize problems and frame solutions, communication skills [and] leadership potential.” It should also take experience into consideration, he added.
Pruitt agreed, noting that “organizations should be measuring a combination of knowledge, technical skills and soft skills, and the ability to combine them."
Technical skills might include familiarity with security tools and offensive and defense techniques, while soft skills include communication, teamwork and creativity. But all too often organizations don't actively test hires for these qualified, "as they are simply much harder to assess," said Pruitt. "Unfortunately, security leaders often discover that their team members lack these skills when they confront their first incident. Ideally, we should assess and screen these individuals in advance.”
On the other end of the spectrum, Downs requires anyone applying to join his team at ISACA to complete several labs and challenges. These exercises “provide me with specific feedback as to their level of technical competence in the five domains of cybersecurity: identify, protect, detect, respond and recover,” said Downs. “I then use this information, in combination with the interview experience, to ascertain the aptitude of the applicant and to determine if they are a good fit.”
With that said, however, a well-thought-out job interview process and a thorough resume review can sometimes help compensate for a lack of testing. “When I conduct an interview, I do pose specific technical questions that only experienced professionals will be able to answer appropriately,” said Downs. “Additionally, thanks to the certification qualifications that many jobs implement, a certain level of assurance can be assumed when evaluating applicants.”
Another advantage to putting cyber job candidates through tests or sims is to determine how they handled themselves in high-stress situations. “We see that withstanding pressure during a security incident can be acquired and improved by repeating these stressful situations by means of simulation, just as you would do in other high-stress roles, such as military pilots,” said Pruitt.
“It is important to understand if someone can work well under pressure,” agreed Downs. “However, oftentimes that aspect of an applicant reveals itself during the interview itself. Having technical questions thrown at you isn’t necessarily a fun experience during an interview. Candidate reactions to these sudden questions tell me a lot about how they will act and react under pressure.”
Combs agreed that observing candidates under pressure can be helpful to “to a degree,” but cautioned that it's “not a definitive gauge. People respond differently to tests than real life situations, especially in an exam environment. I think stress testing in training is extremely valuable to building great teams, but less valuable when it comes to hiring decisions.”
As to whether a lack of testing during the hiring process definitively results in less qualified, that’s a tough theory to prove. However, companies that don’t engage in this practice are at the very least depriving themselves of an opportunity to more thoroughly size up whom they are hiring.
“Companies who invest in their recruiting processes build a stronger talent brand, fill positions faster and retain employees longer than companies that don't,” said Combs.
Still, “the jury is still out” on the value of these job candidate assessment services, he added. As things currently stand, “The majority of internal talent acquisition recruiting functions I've seen are too understaffed and underfunded to really make use of an add-on assessment service.”
