The software supply chain is, as most of us know by now, both a blessing and a curse.
It’s an amazing, labyrinthine, complex (some would call it messy) network of components that, when it works as designed and intended, delivers the magical conveniences and advantages of modern life: Information and connections from around the world plus unlimited music, videos, and other entertainment, all in our pockets. Vehicles with lane assist and accident avoidance.
Home security systems. Smart traffic systems. And on and on.
But when one or more of those components has defects that can be exploited by criminals, it can be risky and dangerous. It puts the entire chain in jeopardy. You know — the weakest link syndrome. Software vulnerabilities can be exploited to disrupt the distribution of fuel or food. It can be leveraged to steal identities, empty bank accounts, loot intellectual property, spy on a nation, and even attack a nation.
So the security of every link in the software supply chain is important — important enough to have made it into a portion of President Joe Biden’s May 2021 executive order, “Improving the Nation’s Cybersecurity” (also known as EO 14028).
It’s also important enough to have been one of the primary topics of discussion at The 2022 RSA conference in San Francisco. Among dozens of presentations on the topic at the conference was “Software supply chain: The challenges, risks, and strategies for success” by Tim Mackey, principal security strategist within the Synopsys Cybersecurity Research Center (CyRC).
Challenges and risks
The challenges and risks are abundant. For starters, too many organizations don’t always vet the software components they buy or pull from the internet. Mackey noted that while some companies do a thorough background check on vendors before they buy — covering everything from the executive team, financials, ethics, product quality, and other factors to generate a vendor risk-assessment score — that isn’t the norm.
“The rest of the world is coming through, effectively, an unmanaged procurement process,” he said. “In fact, developers love that they can just download anything from the internet and bring it into their code.”
While there may be some regulatory or compliance requirements on those developers, “they typically aren’t there from the security perspective,” Mackey said. “So once you’ve decided that, say, an Apache license is an appropriate thing to use within an organization, whether there are any unpatched CVEs [Common Vulnerabilities and Exposures] associated with anything with an Apache license, that’s somebody else’s problem. There’s a lot of things that fall into the category of somebody else’s problem.”
Then there’s the fact that the large majority of the software in use today — nearly 80% — is open source, as documented by the annual “Open Source Security and Risk Analysis” (OSSRA) report by the Synopsys CyRC.
Open source software is no more or less secure than commercial or proprietary software and is hugely popular for good reasons — it’s usually free and can be customized to do whatever a user wants, within certain licensing restrictions.
But, as Mackey noted, open source software is generally made by volunteer communities — sometimes very small communities — and those involved may eventually lose interest or be unable to maintain a project. That means if vulnerabilities get discovered, they won’t necessarily get fixed.
And even when patches are created to fix vulnerabilities, they don’t get “pushed” to users. Users must “pull” them from a repository. So if they don’t know they’re using a vulnerable component in their software supply chain, they won’t know they need to pull in a patch, leaving them exposed. The infamous Log4Shell group of vulnerabilities in the open source Apache logging library Log4j is one of the most recent examples of that.
Keeping track isn’t enough
To manage that risk requires some serious effort. Simply keeping track of the components in a software product can get very complicated very quickly. Mackey told of a simple app he created that had eight declared “dependencies” — components necessary to make the app do what the developer wants it to do. But one of those eight had 15 dependencies of its own. And one of those 15 had another 30. By the time he got several levels deep, there were 133 — for just one relatively simple app.
Also, within those 133 dependencies were “multiple instances of code that had explicit end-of-life statements associated with them,” he said. That means it was no longer going to be maintained or updated.
And simply keeping track of components is not enough. There are other questions organizations should be asking themselves, according to Mackey. They include: Do you have secure development environments? Are you able to bring your supply chain back to integrity? Do you regularly test for vulnerabilities and remediate them?
“This is very detailed stuff,” he said, adding still more questions. Do you understand your code provenance and what the controls are? Are you providing a software Bill of Materials (SBOM) for every single product you’re creating? “I can all but guarantee that the majority of people on this [conference] show floor are not doing that today,” he said.
But if organizations want to sell software products to the U.S. government, these are things they need to start doing. “The contract clauses for the U.S. government are in the process of being rewritten,” he said. “That means any of you who are producing software that is going to be consumed by the government need to pay attention to this. And it’s a moving target — you may not be able to sell to the U.S. government the way that you’re used to doing it.”
Even SBOMs, while useful and necessary — and a hot topic in software supply chain security — are not enough, Mackey said.
Coordinated efforts
“Supply chain risk management (SCRM) is really about a set of coordinated efforts within an organization to identify, monitor, and detect what’s going on. And it includes the software you create as well as acquire, because even though it might be free, it still needs to go through the same process,” he said.
Among those coordinated efforts is the need to deal with code components such as libraries within the supply chain that are deprecated — no longer being maintained. Mackey said developers who aren’t aware of that will frequently send “pull requests” asking when the next update on a library is coming.
And if there is a reply at all, it’s that the component is end-of-life, been end-of-life, and that the only thing to do is move to another library.
“But what if everything depends on it?” he said. “This is a perfect example of the types of problems we’re going to run into as we start managing software supply chains.”
Another problem is that developers don’t even know about some dependencies they’re pulling into a software project, and whether those might have vulnerabilities.
“The OSSRA report found that the top framework with vulnerabilities last year was jQuery [a JavaScript library]. Nobody decides to use JQuery, it comes along for the ride,” he said, adding that that is true of others as well, including Lodash (a JavaScript library) and Spring Framework (an application framework and inversion of control container for the Java platform). “They all come along for the ride,” he said. “They’re not part of any monitoring. They’re not getting patched because people simply don’t know about them.”
Building trust
There are multiple other necessary activities within SCRM that, collectively, are intended to make it much more likely that a software product can be trusted. Many of them are contained in the guidance on software supply chain security issued in early May by the National Institute of Standards and Technology in response to the Biden EO.
Mackey said this means that organizations will need their “procurement teams to be working with the government’s team to define what the security requirements are. Those requirements are then going to inform what the IT team is going to do — what a secure deployment means. So when somebody buys something you have that information going into procurement for validation.”
“A provider needs to be able to explain what their SBOM is and where they got their code because that’s where the patches need to come from,” he said.
Finally, Mackey said the biggest threat is the tendency to assume that if something is secure at one point in time, it will always be secure.
“We love to put check boxes beside things — move them to the done column and leave them there,” he said. “The biggest threat we have is that someone’s going to exploit the fact that we have a check mark on something that is in fact a dynamic something — not a static something that deserves a check mark. That’s the real world. It’s messy — really messy.”
How prepared are software vendors to implement the security measures that will eventually be required of them? Mackey said he has seen reports showing that for some of those measures, the percentage is as high as 44%. “But around 18% is more typical,” he said. “People are getting a little bit of the message, but we’re not quite there yet.”
So for those who want to sell to the government, it’s time to up their SCRM game. “The clock is ticking,” Mackey said.
Click here to find more Synopsys content about securing your software supply chain.
