Back to newsletter

Previous Issue's Dilemma:

How should we automate our security tests?

Since security tests are expensive and time consuming, my company is starting to consider the idea of automating these tests. What do you do to automate the testing process to get a minimum of security? Any information you have about this subject is much appreciated. Which tools do you use? And by the way, how many people are involved? Thanks in advance for your suggestions.

-- C. G. (name and company withheld on request)

Three key steps toward automation

Readers agree unanimously that automated testing is a logical and efficient step in the development process. One anonymous reader says, "Automated testing tools have proven to be very effective in general application regression testing in our organization. We have been using them with a great deal of success on several key Web applications."

Other readers also have had success moving to automated security tests. They provide three steps you can take toward successfully automating your tests:

• Start with a requirements definition.

• Budget wisely for automated testing.

• Manage security issues.

Start with a requirements definition

Complimentary White Paper A CISO's Guide to Application Security Security breaches are expensive. They cost time and effort in litigation, remediation, and reputation restoration. But when it comes to security, businesses rarely do it right. Learn why application security is more critical than ever to your business, and the six steps to secure applications. Download this complimentary white paper.

"Once you define your requirements, you'll know which features you need to look for in selecting a tool for your company. You will also have a better idea of the number of people involved. Finally, the minimum level of security depends on your company's investment in security."

Budget wisely for automated testing

If you're not using automated regression testing tools, then manual testing could, as you have stated, tie up a large number of expensive resources. M.B. adds, "Because of the time manual testing takes, it could be difficult to deliver changes successfully on short notice, and your organization could end up shipping software that hasn't been sufficiently re-tested after changes, sometimes with grim consequences."

M.B. recommends that you budget for three items:

1. Automated tools. Automated testing tools aren't free and can be a serious investment from the outset, even though they will save your organization money in the long run.

2. Training. You will have to thoroughly train the quality assurance developers in using the tools before you can begin to see the payback.

3. Test scripts. Creating adequate test scripts is of utmost importance. If you don't designate adequate resources to early test script development, you may not fully address software unknowns.

Manage security issues

In his response, Charles Le Grand, founder of CHL Global Associates, focuses primarily on how to deal with management when it comes to risk. He offers several suggestions.

• Get a security professional on board. "On the subject of information security, it is tough to ensure management actually makes an informed decision because they typically have little or no understanding of the subject and are often ‘afraid' of it. That said, a security professional (preferably a certified professional with an obligation to adhere to a code of ethics) must state the case clearly to management, quantify to the extent possible the potential extent of harm that can result from the risk, and make it clear how easy it would be for an inside or outside attacker to exploit the known vulnerabilities."

• Make a risk assessment. "If you have internal or external auditors, it will be important to get their assessment of the risk. However, if they are not technically competent, you may need to inform them rather than seek their opinion. You must also consider whether management fosters an environment of openness and honesty with auditors. If they do not, you must consider the organization's culture as a primary reason for accepting unacceptable risks. If your organization is regulated, it also has an obligation for openness and honesty with the regulatory examiners. But again, the culture issue coupled with the extent of regulatory involvement and regulator competence can contribute to your frustration with the unmitigated risk."

• Validate your concerns about risk. "Finally, state your professional opinion that the level of risk is unacceptable, given what you know about the company and technology involved. If you are not certified, it may be worthwhile to get a certified professional to attest to the validity of your concern. Then, if management still decides to accept rather than remediate the risk, they or you must document the decision not to act. The decision could be legitimate. Management may have bigger concerns to address and resource limitations that put your concerns below the level of possible affordable actions."

• Don't bluff. What if management doesn't listen? "If your professional ethics will not allow you to accept the decision, then you can try again, or you can state the problem as the reason for your decision to leave the organization. Make it clear that their decision has put the organization at a level of risk you consider unacceptable. But don't bluff. If you are not prepared to leave, do not play the resignation card. And if your organization is in the position of potentially betraying a public trust by accepting a known risk, you may consider the option of being a whistle-blower. But beware, not all whistle-blowers are treated kindly for their acts, and some have suffered greatly and/or found themselves subject to legal action -- potentially as a scapegoat."

Le Grand finishes his suggestions with this advice: "It is a crazy, mixed-up world. Pick your battles, and plan your strategies wisely. Do your best, and make it clear in your words and actions that you will act only in a professional manner."