Build It, Break It, Fix It: Contesting Secure Development

Typical security contests focus on breaking or mitigating the impact of buggy systems. I will present the Build-it, Break- it, Fix-it (BIBIFI) contest, which aims to assess the ability to securely build software, not just break it. I will also present qualitative and quantitative analysis of data gathered from three runs of the contest, which demonstrates some interesting trends.

In BIBIFI, teams build specified software with the goal of maximizing correctness, performance, and security. The latter is tested when teams attempt to break other teams’ submissions. Winners are chosen from among the best builders and the best breakers.

BIBIFI was designed to be open-ended—teams can use any language, tool, process, etc. that they like. As such, contest outcomes shed light on factors that correlate with successfully building secure software and breaking insecure software. During 2015 we ran three contests involving a total of 116 teams and two different programming problems. Quantitative analysis from these contests found that the most efficient build-it submissions used C/C++, but submissions coded in a statically-typed language were less likely to have a security flaw; build-it teams with diverse programming-language knowledge also produced more secure code. Shorter programs correlated with better scores. Break-it teams that were also build-it teams were significantly better at finding security bugs.

Michael W. Hicks is a professor in the Computer Science department at the University of Maryland, College Park, and is affiliated with the Maryland Cybersecurity Center (MC2). His research focuses on using programming languages and analyses to improve the security, reliability, and availability of software. Noteworthy among his research accomplishments is the development of analysis and compilation tools for enabling software to be safely updated while it runs. He has explored the design of new programming languages and analysis tools for automatically discovering or remediating software flaws and security vulnerabilities. He has recently been exploring new approaches to privacy preserving computation.