A new Google program aimed at continuously fuzzing open source software has already detected over 150 bugs.
The program, OSS-Fuzz, currently in beta mode, is designed to help unearth programming errors in open source software via fuzz testing. Fuzz testing, or fuzzing is when bits of randomly generated code is inputted into programs as a means to discover code and security flaws.
Chrome security engineers Oliver Chang and Abhishek Arya, Google software engineers Kostya Serebryany and Mike Aizatsky, and Meredith Whittaker, who leads Google’s Open Source Research group, announced the project last Thursday.
Our fuzzing-as-a-service for opensource software is now in beta! https://t.co/wYPxBNeEgO
— Kostya Serebryany (@kayseesee) December 1, 2016
The program was developed with help from the Core Infrastructure Initiative, a Linux Foundation collaborative that counts Cisco, Facebook, and Microsoft among its members.
“Open source software is the backbone of the many apps, sites, services, and networked things that make up ‘the internet.’ It is important that the open source foundation be stable, secure, and reliable, as cracks and weaknesses impact all who build on it,” the engineers wrote Thursday, “OSS-Fuzz’s goal is to make common software infrastructure more secure and stable by combining modern fuzzing techniques with scalable distributed execution.”
The project is built on fuzzing engineers such as libFuzzer, sanitizers, AddressSanitizer and a distributed fuzzing infrastructure that catalogs fuzz statistics called ClusterFuzz.
The program has identified bugs in projects pcre2, libchewing and FFmpeg so far. Even more impressive is that Google claims OSS-Fuzz is cranking out four trillion test cases a week.
Engineers say FreeType, an open source library that’s used to display text, is a perfect example of what OSS-Fuzzing can achieve. One of FreeType library’s developers, Werner Lemburg, adopted OSS-Fuzz early on. In October, after a heap buffer overflow was identified in the library, OSS-Fuzz notified the maintainer, who went on to fix the bug. It was confirmed fixed by OSS-Fuzz the same day, Google says.
While the program is in its infancy, developers and open source proponents have lauded Google for the program.
Alex Gaynor, who writes a lot of open source code and previously served as the director of the Python Software Foundation and the Django Software Foundation, tested OSS-Fuzz late last week and called the experience “extremely good.”
“I definitely think it’s something that every OSS project should take a look at,” Gaynor told Threatpost Monday.
Gaynor, who penned a blog entry around his experience on Saturday, used OSS-Fuzz to test libyaml, a C library YAML 1.1 parser and emitter that’s the basis for both Python and Ruby’s YAML libraries. The program ran 17 billion testcases against the library in under a day, according to Gaynor, roughly 30 days of CPU time in less than a calendar day.
While developers have to build fuzzers specific to their project, OSS-Fuzz does most of the work, Gaynor says. He added that in his short experience, sending pull requests for projects is easy enough.
The program will file any bugs it discovers privately and leave a comment when it thinks a crash has been fixed. It makes the bug public seven days after it’s been fixed and even “handles automatically rebuilding when the upstream source changes,” Gaynor points out.
“It was almost no work to write a fuzzing function and get it running, and OSS-Fuzz handles tons of the details around making fuzzing at scale practical; this makes the experience far more pleasant than if I’d jerry–rigged something together myself,” Gaynor, who currently works for the United States Digital Service–part of the Executive Office of the President, wrote.
Gaynor said Monday the only area OSS-Fuzz might need a slight tweak in is its user interface.
“In terms of areas for improvement, the biggest one would be the UI used for looking at stats and crash reports, which needs a bit more polish for non-internal audiences,” Gaynor said.
Open source software experts like Jim Zemlin, executive director at the Linux Foundation, also took time to laud the project on Twitter last week.
This is making the internet more secure with thanks to @mer__edith and Google team. Devs should check this out: https://t.co/kaPfbytvw6
— jzemlin (@jzemlin) December 1, 2016
Now that the program has been announced, Google claims its main focus is fostering OSS-Fuzz usage.
The company is encouraging open source projects, as long as they have a large user base, to join OSS-Fuzz. By doing so, developers would have to subject themselves to Google’s 90-day disclosure deadline but they’d also be joining a rich open source community, the engineers say.
While often viewed as a nuisance, low level bugs like buffer overflow and use-after-free vulnerabilities can have a pivotal impact on software security, especially in the open source realm. Vulnerabilities in libStageFright, a software library coded in C++; part of the Android Open Source Project, led to a series of bugs last year which went on to enable remote code execution and privilege escalation.
The venture is one of the latest efforts from the CII to bolster open source software security. Earlier this year the consortium unveiled a badge program designed to help developers self-certify their projects. The program, which counted GitLab, Node.js, and OpenSSL as early adopters, encourages open source software to follow best practices and self-disclose their security aspects.
The CII was also behind the Open Crypto Audit Project, which was responsible for last year’s TrueCrypt audit and also helped fund the current large scale audit of OpenSSL.
