Assurance and Evaluation: What Next?

ABSTRACTThis talk presents some observations on the nature of system security and security assurance, and on processes that attempt to evaluate assurance. Over the past forty years, approaches to assuring software security have evolved from "penetrate and patch" through attempts to prove security to application of automated tools that help developers detect and remove potential vulnerabilities. A review of that evolution leads to some observations on the effectiveness and practicality of various approaches and to the conclusion that the most theoretically appealing approaches may not be the most practical or likely to succeed. The talk then considers the processes by which user organizations, primarily governments, have attempted to evaluate product security. A review of historic approaches to evaluation supports the conclusion that past evaluation regimes have achieved limited success. The talk suggests some attributes of approaches to evaluation better suited to the realities of processes that achieve security assurance and more likely to provide valuable information to end users.