31 July 2008 One Comment

This is a follow-up article to our initial response to Fortify’s Open Source Security Study. That study, the findings of which were published in an 11 page report, came to three conclusions about the security habits of open source software projects.

The conclusions, all three of them, were just plain wrong. Let’s examine each in detail.

First conclusion:

Few open source projects provide documentation that covers the security implications and secure deployment of the software they develop, a dedicated email alias for users to report security vulnerabilities, or easy access to internal security experts to discuss security issues.

Nothing could be further from the truth. Like we wrote in our first response, virtually all open source projects have some sort of issue tracking and bug reporting system in place. The eleven open source projects that were surveyed by Fortify have those resources. You can verify that visiting the websites of the projects - Derby, Geronimo, JBoss, Struts, OFBiz, OpenCMS, JOnAS, Hipergate, Hibernate, Tomcat, and Resin.

Second conclusion:

Not only did every project that we scanned contain significant security issues, but in all but one, the total number of security issues remained constant or increased between successive releases. This demonstrates that the projects have not adopted a successful secure development process.

Every software application contains “significant security issues”: It’s not unique to open source projects. But if you didn’t know any better before reading this report, you’d come to the erroneous conclusion that only open source software projects do not “adopted a successful secure development process”.

Third conclusion:

Well-known security vulnerabilities, such as Cross-Site Scripting (XSS) and SQL Injection, were among the most common and serious problems identified, which is consistent with OWASP findings. These classes of vulnerabilities can be identified by enrolling in the free Fortify Java Open Review (JOR) project or with open source tools, such as FindBugs. This indicates that the projects do not make use of technology to identify and resolve security issues.

Fortify resorts to an age-old trick - identify a flaw or flaws in a product, and suggest that users utilize their product (Fortify Java Open Review) or a product that they sponsor (Findbugs) to fix it. There’s nothing wrong in pushing their product, but Fortify should have been upfront with its objective. We have no doubt that like closed source applications, that open source software have bugs, but Fortify’s study and accompanying report gives the impressions that open source developers are clueless when it comes to security best practices.

This runs contra to a more technical report by Coverity, published in May, 2008, which showed that open source projects experienced a “16% reduction in static analysis defect density” over the course of a two year period. The Coverity study is an ongoing one and it’s sponsored by the US Department of Homeland Security as part of their Open Source Hardening Project. For the two-year period, more than 250 open source projects were analyzed by Coverity.

As we stated earlier, all software applications have bugs, and developers use different methods to find and fix them, but using sweeping and misleading statements is a dishonest approach to promote usage of your product. The open source community does not like FUD.