The amount of insecure software tied to reused third-party libraries and lingering in applications long after patches have been deployed is staggering. It’s a habitual problem perpetuated by developers failing to vet third-party code for vulnerabilities, and some repositories taking a hands-off approach with the code they host.
This scenario allows attackers to target one overlooked component flaw used in millions of applications instead of focusing on a single application security vulnerability. The real-world consequences have been demonstrated in the past few years with the Heartbleed vulnerability in OpenSSL, Shellshock in GNU Bash, and a deserialization vulnerability exploited in a recent high-profile attack against the San Francisco Municipal Transportation Agency. These are three instances where developers reuse libraries and frameworks that contain unpatched flaws in production applications.
Security researchers at Veracode estimate that 97 percent of Java applications it tested included at least one component with at least one known software vulnerability. “The problem isn’t limited to Java and isn’t just tied to obscure projects,” said Tim Jarrett senior director of security, Veracode. “Pick your programming language.” Gartner, meanwhile, estimates that by 2020, 99 percent of vulnerabilities exploited will be ones known by security and IT professionals for at least one year.
Code Reuse Saves Time, Invites Bugs
According to security experts, the problem is two-fold. On one hand, developers use reliable code that at a later date is found to have a vulnerability. Second, insecure code is used by a developer who doesn’t exercise due diligence on the software libraries used in their project.
“They’ve heard the warnings and know the dangers, but for many developers open source and third-party components can be a double-edge sword – saving time but opening the door to bugs,” said Derek Weeks, vice president and DevOps advocate at Sonatype.
In an analysis of 25,000 applications, Sonatype found that seven percent of components had at least one security defect tied to the use of an insecure software component.
Repositories GitHub, Bitbucket, Python Package Index and NuGet Gallery are essential tools helping developers find pre-existing code that adds functionality for their software projects without having to reinvent the wheel. Java application developers, for example, rely on pre-existing frameworks to handle encryption, visual elements and libraries for handling data.
“Software is no longer written from scratch,” Weeks said. “No matter how new and unique the application, 80 percent of the code used in a software application relies on third-party libraries or components.”
He said enterprises are more reliant on the software supply chain than ever before. But he says many of the go-to open-source repositories that make up that supply chain are not vetted libraries of reliable code. Rather, they are warehouses with a varying percentage of outdated projects with security issues.
According to an analysis of Sonatype’s own Central Repository in 2015, developers had made 31 billion download requests of open source and third-party software components, compared to 17 billion requests the year before. And when Sonatype analyzed its own code library, it found 6.1 percent of code downloaded from its Central Repository had a known security defect.
Weeks says Sonatype’s is doing better than other repositories that offer no tools, no guidance and no red flags to prevent developers from using frameworks with faulty code. “There is no Good Housekeeping Seal of Approval for third-party code.”
“Faulty code can easily spawn more problems down the road for developers,” said Stephen Breen, a principal consultant at NTT Com Security. “Even when development teams have the best intentions, it’s easy for developers working under tight deadlines to not properly vet the third-party code used in their software.”
Breen said when insecure code is unknowingly used to build a component within a software program, problems snowball when that component is used inside other larger components. One example of vulnerable third-party code reused repeatedly is a deserialization flaw in Apache Commons Collections (commons-collections-3.2.1.jar) – first reported in 2015 and patched in November of the same year.
Source: Veracode
Jarrett found there are still 1,300 instances of the old vulnerable version of the Commons Collections lurking inside Java applications using Spring and Hibernate libraries and hosted across multiple open source code repositories.
“The developer knows they are picking Spring or Hibernate for their development project. They don’t take it to the next level and realize they are also getting Common Collections,” Jarrett said. “That Common Collections library is then used by thousands more projects.”
According to Veracode, Apache Commons Collections is the sixth-most common component used in Java applications. It found that the unpatched versions of the software was in 25 percent of 300,000 Java applications scanned. Even more challenging for developers is updating those applications that are using the vulnerable version of libraries and frameworks since flaws were patched.
“Think of it like a faulty airbag. Carmakers used those faulty airbags in millions of vehicles. Now it’s the carmaker on the hook to fix the problem, not the airbag maker,” Jarrett said.
Leaky Apps, Bad Crypto, Injection Flaws Galore
Veracode said the Apache Common Collection example is the tip of the iceberg. When Veracode examined vulnerabilities tied to insecure code it found application information leakage, where user or application data can be leveraged by an attacker, is the most prevalent type of vulnerability, accounting for 72 percent of third-party code flaws. Second are cryptographic issues representing 65 percent of vulnerabilities. That was followed by Carriage Return Line Feed (CRLF) injection flaws and cross site scripting bugs.
Source: Veracode
Compounding the problem is an increased dependency on open-source components used in a wide variety of software products. The federal government is typical. It has an open-source-first policy as do many private companies. Relying on third-party libraries shortens development time and can improve the safety and quality of their software projects, Weeks said.
“Not only does code reuse save time but it also allows developers to be more innovative as they focus on creating new functionality and not writing encryption libraries from scratch,” Weeks said. Done correctly, code reuse is a developer’s godsend, he said.
For those reasons, security experts say it’s time for the industry to stop and consider where code originates. Sonatype, which markets and sells code verification services, promotes the idea of documenting software’s supply chain with what it calls a “software bill of materials.” That way developers can better scrutinize open-source frameworks before and after they are used; making it easier to update those applications that are using vulnerable old versions of libraries.
Sonatype said it found one in 16 components it analyzed had a vulnerability that was previously documented, verified and with additional information available on the Internet. “I can’t imagine any other industry where it’s okay that one in 16 parts have known defects.”
The problem is that among developers there is a mix of denial and ignorance at play. “Developers choose component parts, not security,” Weeks said. It should be the other way around.
“If we are aware of malicious or bad libraries or code, of course we want to warn our users,” said Logan Abbott, president of SourceForge, a software and code repository. “We scan binaries for vulnerabilities, but we don’t police any of the code we host.”
Repositories Say: ‘We’re Just the Host’
Repositories contacted by Threatpost say their platforms are a resource for developers akin to cloud storage services that allow people to store and share content publicly or privately. They don’t tell users what they can and cannot host with their service.
They say rooting out bugs in software should be on shoulders of developers – not repositories. Writing good vulnerability-free code starts at getting good code from healthy repositories with engaged users.
“Bitbucket is to a developer like Home Depot is to a carpenter,” said Rahul Chhabria, product manager for Atlassian Bitbucket. “We’ve built a hosting service with a variety of tools to help developers execute on their vision.”
Chhabria said Bitbucket offers a range of tools to help sniff out bad or insecure components such as the third-party tool SourceClear for scanning dependency chains. It also offers Bitbucket that it says allows for team development of software projects and simplifies peer review. Another features, Bitbucket Pipelines, is also designed to help developers ship high quality code.
GitHub is one of the largest repositories; it hosts 49 million public and private projects for its 18 million users. It does not scan or red flag insecure code hosted on its platform, according to Shawn Davenport, VP of security at GitHub. Instead developers can use third party-tools such as Gemnasium, Brakeman and Code Climate for static and dependency analysis.
“There is a lot of hidden risk out there in projects,” Davenport said. “We do our best to make sure our developers know what tools are available to them to vet their own code.” He estimates a minority GitHub developers take advantage of software scanning and auditing tools. “Unfortunately security isn’t a developers first priority.”
Other repositories told Threatpost they intentionally take a hands-off approach and say expecting them to police their own software isn’t feasible, not part of their mission and nothing they plan to do. They point out, flawed or not, developers want access to all code – even older components.
“An implementation of a library in one framework might not be a security risk at all,” Breen said. He points out developers often temporarily revert to those old libraries as stopgaps should an updated version break a project.
Automated Scanning to the Rescue?
One attempt at nipping the problem at the bud is the used of automated security vulnerability and configuration scanning for open source components. By 2019, more than 70 percent of enterprise DevOps initiatives will incorporate automated scanning, according to Gartner. Today only 10 percent of packages are scanned.
The Node.js Foundation, an industry consortium designed to promote the Node.js platform, relies on a more community-based approach via the Node.js Security Project. The goal is to provide developers a process for discovering and disclosing security vulnerabilities found in the Node.js module ecosystem. According to Node.js the approach is a hybrid solution that consists of a database of vulnerabilities and a community communication channel for vetting and disclosing vulnerable code.
“It’s not a story about security professionals solving the problem, it’s about how we empower development with the right information about the (software) parts they are consuming,” Weeks said. “In this case, the heart of the solution lies with development, and therefore requires a new approach and different thinking.”
