New Study Shows Nearly No Difference in Security of Web Frameworks

A new study by a Web security firm has found that despite the myriad differences in the common programming languages and frameworks deployed on the Web today, there is virtually  no difference in their practical security and resistance to attack.

A new study by a Web security firm has found that despite the myriad differences in the common programming languages and frameworks deployed on the Web today, there is virtually  no difference in their practical security and resistance to attack.

The study, done by WhiteHat Security and based on statistics gathered from nearly 1,700 sites owned by the company’s customers, shows that the choice of Web development environment is just one of many elements that contributes to the overall security of a site–and may not even be near the top of the list of important factors.

“Technically speaking, one Web programming language / development framework can be made basically just as secure (or not) as any other. Empirically, languages / frameworks only show moderately varying security characteristics, such as the classes of attack to which they are
most prone, how long they tend to be vulnerable, and so on. From a security posture perspective however, the languages / frameworks appear more the same than different. As such, we concluded that an organization’s chosen website development technology alone does not guarantee application security success (or failure) — many other factors must be considered,” the study concludes.

“That was the really big takeaway for me. They’re all pretty bad once they get into production, and the difference when they get into production isn’t great,” said Jeremiah Grossman, the CTO of WhiteHat. “Some of these languages and frameworks have more security mechanisms built in than others do, but it’s questionable whether people use them in the field. There’s a difference between theory and practice.”

WhiteHat, which specializes in helping enterprises secure their Web applications, analyzed data on a number of the more popular Web development frameworks and languages, including Microsoft .NET, ColdFusion, Struts, Perl and others. The information on which applications had which vulnerabilities, how long those vulnerabilities were open and how severe the flaws was collected from the thousands of sites owned by WhiteHat’s customers.

The company found that the distribution of certain kinds of vulnerabilities across the spectrum of languages and development frameworks was quite interesting. For example, more than 80 percent of sites written in Perl contained at least one cross-site scripting vulnerability, while the number of .NET sites with the same kind of flaw was about 50 percent.

Unsurprisingly, WhiteHat’s data shows that enterprises are taking their time when it comes to remediating vulnerabilities in their Web applications. The company looked at data from the last 12 months on the most common classes of vulnerabilities identified and fixed in Web applications. Remediation of common flaws, such as XSS or SQL injection, can take days, weeks or even months, the study found.

“Across essentially all classes of attack, remediation requires weeks to months. In today’s threat landscape, where most incidents are Webbased, this average response time is not nearly fast enough. This signifies a need for organizations to improve their capability to monitor incoming attacks and respond to verified vulnerabilities. Unless a software security development lifecycle effort can guarantee perfection, which it can’t, organizations must be prepared to react
appropriately without being operationally disruptive. We have seen several cases where issues have been fixed within a 24 to 48 hour window, through prioritization and use of tools like Web Application Firewalls, demonstrating that it is possible,” WhiteHat found.

Grossman said much of this problem can be put down to the competition for internal resources in enterprise IT departments.

“There’s a bargaining for resources that takes place,” he said. “A lot of these applications are custom code, so there aren’t patches available for them. They have to develop the patches internally and they have to negotiate for development resources. The questions is, do we spend time on pushing this potentially money-making feature or do we develop a patch for a flaw that may not get exploited? The security teams have to educate the development groups, and there’s a knowledge and respect gap, as well.”

Suggested articles


  • Andrew on

    I apologize, but how can this "research" be an accurate measure of what it claims to provide?

    The research is based on data gathered from the clients of a company which was hired to audit & protect vulnerable websites.  So, to say that of all the sites we've audited and protect, we've found they are all basically vulnerable in different ways, is kinda weak sauce. 

    If the companies had strong security development programs why would they be asking for help in the first place? They'd be addressing these things in QA before they ever made it to production.  So, how then can you reasonably extrapolate from the data gathered, such a generally bold statement that all website frameworks are basically the same when all you've only audited ones with issues?  (You can tell these clients have broken development processess if it takes them "days, weeks or even months" to fix XSS vulnerabilites or SQLi)

    It's kind of like claiming that, regardless of the reasons people go to the hospital, we have concluded they are all basically sick.  So what?

  • Jeremiah Grossman on

    @Andrew fair question, I'll do my best to answer. To clarify, we are hired to monitor and measure the security of websites via vulnerability assessments -- not so much "protect" because we can't fix the issues for them. Secondly our data was meant to see if websites using a particular development technology had substantively better security than another. We found no such evidence. Lastly, let me assure you the sample set of websites is sufficiently diverse for such analysis, many of which do indeed have mature development programs and test during QA. They have vulnerabilities just like everyone else, but in those cases what sets them appart is that they fix them. At this point one might hypothesize that process is much more effective and yielding secure websites than development technology.


  • Christopher E. Stith on

    Security is always a process. Even if you develop perfect security and abandon change, someone will find a way to make you vulnerable again. Apathy is the truest road to insecurity. As  Wendell Phillips, Andrew Jackson, and many others have said, eternal vigilance is the price of liberty.

    Liberty, after all, is being secure from the control of others. So by extrapolation, the price of security is eternal vigilance. You can never really be free without security, and you can never really be secure without liberty. This is as true in putting your software out in the public sphere as it is in any other portion of your life. Control access to your person, your data, your home, and your immediate surroundings wherever you go or you are niether truly free nor truly secure. Lack of attention to how others want to access or control you or your belongings is always the easiest way for them to gain control from you.

  • AppSec on

    @Jeremiah: "Some of these languages and frameworks have more security mechanisms built in than others do, but it's questionable whether people use them in the field. There's a difference between theory and practice."

    Based on this statement, doesn't the research truly indicate that developers aren't developing secure sites regardless of the framework?  In order to determine whether or not the frameworks offer the same security, wouldn't you have to compare sites which have taken advantage of the security features available?

  • Jeremiah Grossman on

    @AppSec As we know from statistics, correlation is not the same as causation. What we can say, backed up by data in this report, is that a "secure" website does not really correlate to any particular programming language or development framework. As we tried to dig into with our last report, we had a close look at the websites with zero vulnerabilities -- which is not to say they didn't at one time have them. What we did find is that they started off with less vulns and actually fixed their issues. From the outside this seemed to indicate just a sincere business desire to be secure. People, process and technology are just a byproduct of that desire.

  • Juan C Calderon on

    This comes to reinforce the fact that App Sec is also (still) a technology problem, Frameworks provide more and more security features over time, but they make them all optional for the developer to implement. So if the developer is not aware os simply does not have enough time for "optional" features they will not be implemented.

    Framework developers.... making available a rich set of (optional) security features in a framework does not makes applications secure by default, here is where the "There's a difference between theory and practice" cames to scene.

    On the other side, I personally categorize technologies by how "easy to secure" they are, to help people take desicions on what technology to use. Notice that ALL TECHNOLOGIES ARE SECURABLE, but is is not the same effort to secure a CGI in BSH or C (Hard to Secure - no sessions, no built-in validation, etc), a Perl or PHP application (Medium to secure - Sessions, manual or plugin-based validation, no  built-in authentication/Authorization, etc), or a Java/Struts/Spring or .NET App (Easy to secure - Built-in Authentication, Authorization, Validation, Good Error Handling, etc) just to mention a few examples.

    So my personal recommendation is, as frameworks do not enforce security, at least you can choose a technology/framework that will take less effort to fix, and push for the industry to really move and build security by default rather an as optional skippable features.

  • Jeremiah Grossman on

    @Juan -- FTW! opt-in security is nice, but doesn't get us to the place we need to be. At least orgs and dev manages can know that there is no "wrong choice" when ti comes to framework selection that might prevent them from being "secure."

  • Francesco on

    Interesting feed. By the way, you also never mention the fact that companies usually refer to specialists for their security issues and testing and they are generally not assured that the same level of accuracy will be obtained for code reviews against a technology or another, so i'd rather suggest companies to spend money on SDLC for future projects instead of develop and review after.

  • Josh on


    > "What we can say, backed up by data in this report, is that a "secure" website does not really correlate to any particular programming language or development framework"

    I'm curious why this is an interesting result.  Would not a much more interesting question be "is there a correlation between a particular programming language and how time intensive it is to create a secure website".  The conclusion that "Technically speaking, one Web programming language / development framework can be made basically just as secure (or not) as any other" is basically just a glorified way of saying that most of the languages/frameworks don't make it technically impossible to prevent a vulnerability, but that conclusion does not offer any real knowledge.  People write GUI apps in Perl - it isn't impossible, just a waste of time.

    If I am a Java guy considering Struts versus Spring and one lets me make a secure website faster than the other that is useful information; I don't care that a competent developer could make a secure website with either, but rather what the development effort delta between the two happens to be. 

    Your methodology and results don't actually tell me anything about whether it benefits my organization to use one framework/language in place of another, only that organizations tend to produce comparable products regardless of tools at their disposal.  I don't just care if the end products are comparable, but if the process to get to the end products gets me to the end product more efficiently, and that latter factor is much more important from a business decision point of view.  Competent developers could technically make a secure and robust website using a C-cgi but we don't write them anymore because C is a waste of developer time relative to more modern technologies.  Is the process of producing a website securely more efficient with one language rather than another, not is the output the same.  Your paper really seems to say that no language really compensates for lack of developer skill and knowledge, but my organization is in serious trouble if I think a language is the answer for that problem.

  • Anonymous on

    I would like to know which websites are really taken for this audit and research. Are they some of top banking or other website where security is one of the prime importance. I know hacking possiblities are there for every website but nowadays it has too tough for hackers.


    <a href="">Nathan Thomas</a>

Subscribe to our newsletter, Threatpost Today!

Get the latest breaking news delivered daily to your inbox.