I was at an information security holiday party that was combined with an open forum discussion on security trends, current interests, and general musings. As always we had a lively discussion on everything from PGP to PCI. The one discussion that really got my interest was the discussion on penetration testing. Honestly, I’m not sure if there’s a more confusing term within information security then penetration testing. A decade ago, at least for me, it was pretty clear what it meant. Today? I don’t know.
Okay, so I know what penetration testing is, what has changed is the client’s understanding of what it is they need. As an example, a consultant from another company mentioned that a client had two different companies do testing for them. Their company did a true penetration test while the other company did automated scanning. As you can imagine one company charged much more while the other was relatively inexpensive, definitely an apple to orange comparison. So upon delivery the client asks the questions, “Why did they find so many more vulnerabilities and they were much less expensive?” At that point it’s an uphill battle to justify expenses and validity of services. You know what happened; the company that ran automated scanning had lots of false positives that they didn’t remove. The company that did automated scanning and followed it up with manual testing removed all the false positives. Most likely the depth of vulnerabilities and the value of the true penetration testing were far better than merely automated scanning.
Most of the problem, I believe, has to do with us as consulting companies. We use terms such as ethical hacking services, penetration testing, scanning, etc. interchangeably. Most of us know what each means, where we fall short is defining what the client means. They rely on us to help them define that need, so we need to have them explain clearly what they need. Instead we often take the path of least resistance in order to meet what the client is asking for, not what they need.
All of this could be changed if we took the time to discuss desired outcomes, reason for testing, and whatever other reasons they’re requesting penetration testing. There are some clients that only want an automated scan and there are some companies that only provide automated scanning. What we need to do is to stop calling that penetration testing.
When people say they need a pen test we shouldn’t assume anything, we need to start asking questions. We should assume that someone has gotten a hold of them and bastardized the meaning of penetration testing. If what they believe to be a pen test is actually just automated scanning it’s better for everyone to figure that out early. That way you can educate and help them to know what they should expect for the level of work.
So what happened to the guy I mentioned above that did a pen test while the other company did automated scanning? He didn’t take the time to articulate the value of a true pen test upfront and when he tried to explain it on the back end he appeared to lack credibility with the client.