Fossology

To help celebrate Fossology’s 10th anniversary, we look at how the project makes it easier to understand and comply with open source licenses.

FOSSology turns ten this year. Far from winding down, the open source license compliance project is still going strong. The interest in the project among its thriving community has not dampened in the least, and regular contributions and cross-project contributors are steering it toward productive and meaningful iterations.

An example is the recent 3.2 release, offering significant improvements over previous versions, such as the import of SDPX files and word processor document output summarizing analysis information. Even so, the overall project goal remains the same: to make it easier to understand and comply with the licenses used in open source software.

There are thousands of licenses used in Open Source software these days, with some differing by only a few words and others pertaining to entirely different use universes. Together, they present a bewildering quagmire of requirements that must be adhered to, but only as set out in the appropriate license(s), the misunderstanding or absence of which can revert rights to a reserved status and bring about a complete halt to distribution.  

How FOSSology came to be

In short, there are over a 1000 different ways licensing can go mildly or horribly wrong, creating a desperate need to find one single way to make sure everything goes consistently right. Enter FOSSology, which as the website points out, is all about scanning: It’s a framework, toolbox and Web server application for examining software packages in a multi-user environment.

There are several important highlights since the first version of the FOSSology project was published in December 2007. The Linux Foundation started hosting it in 2015. The 3.2 release was in March 2018, which, as mentioned above, provides the ability to import SPDX files. SPDX (Software Package Data Exchange) is another Linux Foundation project that helps reduce complexity by defining standards for reporting and sharing licensing information. FOSSology is the first open source project to consume SPDX in this way.

“This project has been more successful than anticipated, because license compliance was a very special topic, and running it as an open source project is also difficult, because it has a naturally small community,” said Michael C. Jaeger, Senior Research Scientist Open Source Software at Siemens AG, Maintaining FOSSology and SW360, and Trainer at SW Compliance Academy.

When goal and delivery are tightly entwined, as are the benefactors and beneficiaries, good things come from any project.

“License compliance for open source projects is hard, and FOSSology helps here by doing most of the work, such as scanning the files to find licenses, copyright statements and more, to simplify the necessary clearing. It also generates reports which can be used to document the results, which is rather important in the context of larger companies,” said Maximilian Huber, a software consultant at TNG Technology Consulting.

A paper titled “The FOSSology Project: 10 Years Of License Scanning,” has been prepared to commemorate the 10th anniversary. Project members will be participating  at the FSFE’s Legal and Licensing Workshop in Barcelona this week to present on the project.

The project’s value and who benefits  

“It is important because it offers organizations a free software solution for license compliance – an area where commercial products have a very dominant position for more than a decade. However, with free software, especially open source projects can implement license compliance without upfront cost,” explained Jaeger.

FOSSology fits in well with the other open source compliance related projects like SPDX, OpenChain, and SW360. Indeed, there is even community and developer cross-over with some of these projects and FOSSology.

“There is one person who is a maintainer in both the SW360 and FOSSology projects, and there are some persons contributing to both projects in different roles,” said Jaeger. “Consequently and naturally, there is good coordination between both projects. The FOSSology project also has a long history for supporting SPDX since it represents the de facto standard for exchanging license compliance information.”

“With its review functionality, FOSSology was one of the first supporters of the concept of concluding a license in SPDX,” he added. “It was also the first project which allowed for importing SPDX descriptions, another elementary support because the “X” in SPDX stands for exchange and not “eXport.” As far as I know, OpenChain is not concerned very much with particular tooling; however, FOSSology helps to implement OpenChain conformance.”

And the momentum continues. More changes are on the horizon and some new obstacles as well.

“In the future, more and more open source projects will be straightforwardly licensed and the strong scan correction functionality and file review functionality of FOSSology will move to the background,” said Jaeger.

“However, questions still arise because of incompatibilities of licensings, or in considering obligations of licensing. Therefore, FOSSology needs to shift its focus from correcting scan results of not-well-formed licensing to licensing analysis and license problems on the component level.”

Modernization efforts are also under consideration.

“An important goal is to modularize the parts of FOSSology, to allow a smooth transition to a more modern architecture and software stack,” added Huber.

As even more licensing and related tools cross the horizon, simplifying the information exchange between them and FOSSology will be an ongoing task. That in turn will further cement FOSSlogy’s place in the license compliance ecosystem.

But today, all attention is on a decade of successes and the community that’s responsible for so many wins.

Happy anniversary, FOSSology!

software security

Software security requires discipline and diligence, said Mårten Mickos, speaking at the Open Source Leadership Summit.

Achieving effective security takes constant discipline and effort on everyone’s part – not just one team or group within a company. That was Mårten Mickos’s message in his keynote speech appropriately titled, “Security is Everyone’s Responsibility,” at The Linux Foundation’s recent Open Source Leadership Summit (OSLS).  

Mickos, CEO of HackerOne, which he described as a “hacker-powered security company,” told the audience that $100 billion has been spent on cybersecurity, yet, “Half of the money is wasted. We’ve been buying hardware and software and machines and walls and all kinds of stuff thinking that that technology and [those] products will make us secure. But that’s not true.”

Even if you ply your network with hardware to create a perimeter around it, it won’t make your organization any more secure, Mickos said. The answer is much simpler, he maintained, and the magic bullet is sharing.

“You share the defense, you share information, you work together,’’ he said. “You can’t have secure software if just some of your software engineers are in charge of security. You can’t just delegate it or relegate it to a security team. If you do that it won’t happen.”

Mickos likened that approach to the 1990s, when companies had quality managers and people got ISO certifications. “It didn’t help. It reduced quality in the companies, because people felt that quality now was the job of somebody else, not of you.”

Discipline

Software security, Mickos said, “only happens when we’re very disciplined.”

Mickos’ company has 160,000 contributors, including security researchers, ethical hackers and “white hats;” people who have signed up to find flaws in software, he said.  Security vulnerabilities can emanate from situations even when there are no bugs, he noted, adding that HackerOne hacked the U.S. Air Force in eight minutes.

“We found 200 vulnerabilities in the Air Force’s systems, 20 of those were found by Jack Cable, a 17-year-old high school student from Chicago, Ill.,” he said.

HackerOne has fixed over 65,000 security vulnerabilities, Mickos claimed. “So that has removed a lot of holes where criminals could have entered. But there are still tens of millions of vulnerabilities; no one knows the exact number. But if we deploy 100 billion lines of code every year … there’s a lot of security to look after.”

Pooled Defense

In his speech, Mickos promoted the notion of a “pooled defense;” the idea that “the number of defenders is far larger than the number of bad guys.’ He said there are far more white hats in the world than there are cyber criminals or “black hats.”

Cyber threats are often characterized as being asymmetric, he said, in the sense that one single criminal attacker can cause a lot of harm — so much so that a company needs 100 people to defend against it.

“If companies can get together and pool their defense, you … suddenly you have 10 times the power of the attackers,’’ he said. “If you share information, share the defense, share best practices, and share the act of responding to threats, then you overcome the asymmetry and you turn it around.”

It takes discipline and diligence, Mickos said, recalling how Equifax had “so many failures and acts of negligence or … omissions in the way they handle security,” and that “it was one single software vulnerability that led to the data breach in their systems.” Meanwhile, he added, “There’s nobody here who has a software system with just one vulnerability.”

While people often complain about long passwords or having to use multi-factor authentication because it is so time-consuming, they had better get used to it, he cautioned.

“Security doesn’t come for free. The only thing that … acts against these threats is the discipline and diligence [and] remembering long passwords,’’ Mickos said. “Even when somebody invents a method where we don’t need passwords anymore, you will be asked to do something else which is burdensome and every day, and where you’re not allowed to miss it one single time.”

Mickos also had a message for educational institutions: “Don’t call it computer science and software engineering unless there’s security in it. Today, you can graduate in CS without taking a single course in security.” He said he didn’t pay attention to the importance of security when he was in college, but different times call for a different approach. Today, security “has to become part of everything we do.”

We Can Turn the Ship

When everyone recognizes that security is a shared responsibility, he stressed, “the ship will turn. It’s a big ship, so it turns slowly, but it will turn, and we will get to a state that is similar to what we have with airline safety or hospital hygiene or … automotive safety, where today it all works. But it works because we do it together and we jointly take responsibility for it.”

Watch the complete presentation below:

The key to open source compliance is knowing what’s in your code, right down to the exact versions of the components, says Ibrahim Haddad.

Companies across all industries use, participate in, and contribute to open source projects, and open source compliance is an integral part of the use and development of any open source software. It’s particularly important to get compliance right when your company is considering a merger or acquisition. The key, according to Ibrahim Haddad, is knowing what’s in your code, right down to the exact versions of the open source components.

Ibrahim Haddad

Haddad often writes about compliance with the aim of simplifying what can be a complex and daunting process. Recently, Haddad, who is Vice President of R&D and Head of the Open Source Group at Samsung Research America, wrote Open Source Audits in Merger and Acquisition Transactions, a free ebook from The Linux Foundation. In the book, Haddad provides a practical guide to open source audits in merger and acquisition (M&A) transactions and offers tips for improving general open source compliance preparedness. We reached out to Haddad to understand more about the importance of compliance and audits in the open source world.

The Linux Foundation: A common perception is that using open source software means you do not have to worry about negotiating licenses, fees, and other complications associated with proprietary software. So, why should enterprises care about compliance?

Ibrahim Haddad: It is true that open source software has to a large extent simplified the process of software procurement. The traditional procurement model for proprietary software has always been heavy on the front end, as it involves trial and evaluation, negotiation related to possible customizations, licensing terms, fees, and several other factors. With open source, it is still true that you should evaluate the software, compare it to other possible alternatives, and evaluate if the license of that software is in line with how you plan to use it.

However, this is generally the extent of the initial effort. Once you ship a product, you then must demonstrate that you have respected the terms of the licenses attached to the open source components. That may mean providing a written office, publishing all copyright, attribution and license notices, and/or making available source code including any modifications you have introduced.  Obligations will vary based upon the terms of the open source license and how the code is used.

Companies must make open source compliance an engineering priority, as it is the best way to display their fulfillment of the license obligations. If a company is unable to demonstrate compliance and is unwilling to become compliant when challenged, the owners of the copyright on the open source software may decide to revoke the license.  The company could easily end up in a very difficult space where they may need to recall products and re-engineer around the code.

The Linux Foundation:  In my opinion, there are two primary aspects of Open Source — consumption and contribution. What role does compliance play in these cases? 

Haddad: I agree that the two primary aspects of open source are using and contributing. An enterprise can choose open source components and deploy them in their software stack. An enterprise can also decide that certain open source components are strategic to their products and contribute to these components, inject new dynamics in the projects, and steer them in a technical direction that is favorable to their products.

In the first case, compliance is a critical aspect of the “usage” model. A product or software stack that incorporates open source and is being made commercially available must demonstrate open source compliance. This is essential, and no enterprise should risk their profitability with incomplete compliance.

In my mind, contributing involves a different type of compliance than what is implied through the “usage” model. My recommendation and actual practice for code contribution is to follow an internal process that includes:

  1.     Scan source code intended for contribution to identify its origin and license.
  2.     Ensure you have the rights to contribute it under the project’s license.
  3.     Get your company’s approval for that contribution – and in the case of CLA, also getting approval to sign the CLA on behalf of your company before making the contribution.

I will explain my logic for these three steps:

On 1: It is necessary to identify all the code planned for contribution, and any licenses upon it. This step will also help you identify any possible incompatibilities between the licenses of the contribution and the target project. If the code you plan to contribute and the target project have incompatible licenses, or if you discover the code was copied from somewhere else, then you will need resolve the issue before your code submission.

On 2: This step ensures that you are not accidentally open sourcing third-party proprietary code.  This can be a problem in big internal projects with legacy code.

On 3: In most companies, ongoing contributions require approval following whatever internal process that has been set in place. That process should also address who can sign and submit a CLA as an individual contributor (employee of company X), or on behalf all contributors from company X.

Following these steps, you will be able to significantly minimize legal or compliance risks resulting from using or contributing to open source projects.

The Linux Foundation: If you do serve external customers, but you are not shipping any code, then you are simply offering a service. Do you have to be compliant in that case also?

Haddad: I would like to highlight two use cases here. The first is using open source software in your company for internal purposes: IT, testing, evaluation, etc. In such cases, there are no compliance requirements because it is never distributed.

The second case involves offering online (or web) services to clients using software that incorporates open source software. In that specific case, some licenses such as the GNU Affero General Public License (AGPL) do require companies to comply with the license obligations, even if the software itself never changes hands.

Therefore, regardless of how you use the open source software, I am a strong believer in the value of going through a compliance process to identify open source code, applicable licenses and usage models. I believe that good compliance practices are also good engineering practices.

The Linux Foundation: What was the motivation behind writing Open Source Audits in M&A Transactions? Who is the main audience of the book?

Haddad: I mainly had three motivations for writing this book:

  • The first motivation was the lack of documentation around the open source audit process that must take place prior to a merger or an acquisition. If you search online for documentation on this topic, you will not find much outside of the advertisements from various compliance services providers.
  • The second motivation is saving time. I get many inquiries about this process and having such a document is great to share when I am asked about it. In addition, I thought that if I am able to produce a document that highlights the process and explains it, maybe this will encourage others to write about it and share their experiences and recommended practices.
  • The third motivation was related to innovation in the compliance space that was not getting the attention it deserves. For example, the “Blind Audit” model from FOSSID AB where your compliance service provider can complete the audit without having access or even looking at the source code. This level of privacy and security is highly desirable.

The target audience of the book is anyone interested or involved in ensuring open source compliance. Although the ebook focuses on the specific process during an M&A transaction, it offers various recommendations on how to be compliant. These recommendations apply to any company that uses open source. Therefore, if you are an engineer, legal counsel, someone who works in software procurement, or anyone who is involved in open source in their organization, then you will get something out of reading the book.

The Linux Foundation: Compliance can be a challenge when you use a lot of open source projects with many license mixes. What practices would you suggest to companies to help them with their compliance efforts?

Haddad: I think we have come a long way in the open source compliance domain. Back in the early 2000s, open source compliance was really misunderstood, in large part because it was a developing domain. Today, open source compliance is well understood and most companies have a good enough understanding of the licenses and what needs to be done to stay in compliance. My view is that the top challenges are related to scale and building compliance into the software supply chain.

Of the two challenges I just mentioned, scale is the hardest, and it covers multiple dimensions. One is scaling at the project level, when you are dealing with hundreds of open source components and potentially thousands of open source snippets. The other dimension is scaling to address the complexities of a product with an arbitrary number of licenses involved. The way to deal with that is twofold: process and automation. In November 2016, The Linux Foundation published an ebook entirely dedicated to this topic: Open Source Compliance in the Enterprise.  It offers a guide for how best to use open source code in products and services and participate in open source communities in a responsible way.

The second challenge is building trust in the software supply chain, and there are several initiatives that are geared for that purposes. The most prominent projects are OpenChain and SPDX, both hosted at The Linux Foundation. The SPDX initiative has created a standard method to communicate software components, licenses and copyright information associated with the software. In addition, the OpenChain project is offering a systematic approach for companies to build their compliance efforts and be in conformance of various levels of compliance. They are also offering a training curriculum that companies can adopt and customize for their internal use.

For organizations that rely heavily on open source for their products/software stacks, it is essential for them to participate in these efforts. When you look at the errors that lead to non-compliance, we’ve notice that large companies have significantly improved their internal compliance practices. However, many compliance issues are still being propagated via the software supply chain, from upstream suppliers whose compliance practices are not as rigorous. The final product integrator is responsible for compliance obligations even if they did not create the code they distribute, so this is a relevant issue throughout the entire supply chain. Both SPDX and OpenChain help minimize the compliance gap and ensure proper compliance when software changes hands.

The Linux Foundation: What would be your top recommendations for companies who are major consumers of open source software but are not well versed in ensuring compliance?

Haddad: I have one recommendation: know what is in your code, right down to the exact versions of the open source components. This statement sounds simple but it involves setting up a compliance policy and process, investing in tooling and automation, training employees, and assigning someone (or a small team) to oversee the overall effort.

Also remember that open source compliance is an ongoing process, not a destination. Maintaining good compliance practices enables companies to be prepared for a possible acquisition, sale, or product or service release. For this reason, companies are highly encouraged to invest in building and improving upon their open source compliance programs.

There are also many resources available, so please check them out:

  • Open Source Compliance in the Enterprise is a practical guide for enterprises on how to best use open source in products and services in a legal and responsible way.
  • Practical GPL Compliance is a guide for startups, small businesses, and engineers, particularly focused on complying with the versions of the GNU General Public License (GPL).
  • OpenChain Curriculum is designed to help organizations meet the training and process requirements of the OpenChain Specification. It can also be used for general open source training.
  • The Linux Foundation offers a free open source compliance course for developers.

Modern open source projects rarely consist solely of all new code, written entirely from scratch. More often, they are built from many sources. And, each of these original sources may operate under a particular license – which may also differ from the license that the new project uses.

license scanning and complianceA new publication, called License Scanning and Compliance Programs for FOSS Projects, aims to clarify and simplify this process. This paper, written by Steve Winslow from The Linux Foundation, describes the benefits of license scanning and compliance for open source projects, together with recommendations for how to incorporate scanning and compliance into a new or existing project.

Winslow runs The Linux Foundation’s license scanning and analysis service, and he advises projects about licenses identified in their source code and dependencies.

He says that getting license compliance right early can help attract contributors and users to an open source project. However, he notes that license scanning and compliance are not end goals; rather, they are processes that can serve other objectives, including:

  • Protecting the project’s developers.
  • Assisting downstream compliance efforts.
  • Demonstrating project maturity.  

According to Winslow, “any project that implements license scanning and compliance should aim to make it sustainable” and should set realistic goals to avoid being overwhelmed by the number of options and issues that may arise.

Winslow also explains how using tools, such as FOSSology for license scanning and Software Package Data Exchange (SPDX) to help package scan results into meaningful reports, can help projects succeed in compliance efforts.

Learn more and download this free publication now.