Development teams today are under two opposing pressures. On one side, you have development velocity increasing. Tight deadlines for shipping stories that advance the business are top of mind. Teams are deploying new code weekly, daily, and even hourly.
On the other side, security teams are shifting security responsibilities left for developers to manage. Results of security scans on the path to production can block deployments and show up in the developer’s queue for rapid resolution. The pressure is on to fix vulnerabilities and redeploy, completely disrupting the developer’s flow.
This time-consuming work of remediating security vulnerabilities (and keeping code updated to avoid exploits entirely), competes against delivering business value. Addressing vulnerabilities means other work slows down or has to wait. What’s winning in your organization?
I’d suggest that this does not have to be an ‘either–or’ situation. Instead, it can be a ‘yes–and’ for everyone, creating better productivity and collaboration across development and security teams.
In this article, we’ll explore:
- The state of security scanning and remediation today
- How to integrate auto-remediation in the CI/CD process
- How security and platform teams can initiate auto-remediation
- Why auto-remediation is the way forward
Current state of security scanning and remediation in DevSecOps
With today’s DevSecOps practices, application security has been shifting left to developers—with the aim to eradicate security issues at the earliest stages of the software development lifecycle. This is backed by team behavior as noted in a recent DevSecOps survey:
- 74% of security team members said their orgs have either shifted security left or are planning to in the next three years.
- 53% said they are responsible for application security as part of a larger team—up from 44% last year.
Through the use of scanning tools, development teams have access to so many reports and alerts regarding security issues that they are inundated with more information than they can reasonably act on, including many false positives. The rush to release means developers focus remediation efforts on code they are currently working on, leaving code not under active development with insufficient coverage.
In addition, addressing software supply chain security issues continues to confound teams. Third-party code in today’s codebases can create significant security vulnerability exposure. Yet it’s often more than a dependency bump to fix the problem, as one software version upgrade can easily turn into a major framework migration effort with breaking changes.
From the developer’s perspective, they dread getting assigned vulnerabilities. The pipeline can be halted while they deal with unplanned work that requires research and remediation among all the feature development commitments already promised to the business.
From the security professional’s perspective, too many vulnerabilities and bugs are getting through (despite all the scanning), and they have real concern for the future as known vulnerabilities outpace remediation.
To keep up, development teams need more than reporting automation—they need automated code remediation. Let’s focus on two use cases of security scanning where auto-remediation is increasingly required:
- Integrated into the developer workflow and CI/CD processes
- Asynchronous to CI/CD flow, enabling broader coverage
Auto-remediation integrated into the CI/CD processes
It’s common for developers to use static application security testing (SAST) tools early in the software development process to check for security flaws and weaknesses in the source code that could lead to vulnerabilities. Developers can run SAST checks on their code locally and at check-in. SAST scanning and other security vulnerability scanning are commonly an automated part of the CI/CD process as well.
These scanning tools help developers to find application security issues and sometimes offer remediation advice. This continuous feedback is helpful in cleaning up code prior to production deployment, but developers are still doing the work of remediating the source code, a manual, repetitive, time-consuming process. And with release deadlines looming, teams are typically only fixing high-priority issues.
That’s why scanning is no longer enough. If you can find the problem, why not just automate the fix for developers too?
That’s what Moderne, an automated code remediation platform, is able to do. It uses a sophisticated, rules-based engine to search the codebase and actually remediate the source code—fast and accurately.
The platform works by ingesting an advanced data model of the source code, called a Lossless Semantic Tree (LST), and running expert code transformation recipes on the LST to enact code changes. The recipes are programs that are created by a growing ecosystem of open-source software providers.
Developers can run the auto-remediation locally using the Moderne CLI to iterate quickly, such as kicking off the Common Static Analysis Issues recipe in their own console. And recipes can be run as part of the CI/CD pipeline execution as well.
Our partner, Zivra, has done a great job demonstrating how an auto-remediation step can be integrated in a continuous delivery pipeline—to not only identify security issues but also fix them on the spot.
Zivra is a premier IT services firm that provides high-value consulting services to Fortune 1000 clients with the goal of helping them develop and implement a customized IT transformation strategy. Zivra offers DevSecOps Advisory services to collaborate with enterprises in their strategic initiatives around DevSecOps, AI test automation, cloud strategy, and digital transformation. They also provide services for creating transformation recipes that may be custom to your organization.
Figure 2 shows a software delivery pipeline in the Harness CI/CD platform that integrates steps of an automated remediation process to enforce software security and quality. Watch the full demonstration from Zivra here.
The core steps of the automation in this particular demo include:
- When building and packaging the code artifact at compilation time, a parallel ‘ingestion’ step occurs where the LST is built.
- The Moderne agent retrieves the latest LST and uploads it to the Moderne platform for operation.
- The platform runs the Common Static Analysis Issues recipe on the LST to auto-remediate the issues it finds within minutes. In the Moderne platform, you can see the diffs and explanations of the changes.
- If remediations are required, pull requests are automatically created back to source code control and processed through the ticketing system with priority.
- Code changes are reviewed and approved by developers, and the CI/CD pipeline continues.
This integration goes beyond typical scanning solutions. It’s fixing the issues in the source code with 100% accuracy, requiring only approval from development to move forward. This improves code quality and security without eroding delivery efficiency. You can run a wide range of recipes beyond the one shown in this demo to identify and fix issues within the CI/CD process.
Platform and security teams auto-remediating vulnerabilities
Most organizations also are managing application security and compliance with software composition analysis (SCA) tools and resulting software bills of material (SBOMs). The problem remains that security vulnerability remediation is a manual activity that suffers from the diffusion of responsibility. Having each developer prioritize their work to fix identified vulnerabilities can have mixed results (as Figure 1 data indicated).
Moderne can identify vulnerabilities, but then takes things to another level by:
- Looking beyond individual code repositories that a developer is working in to address organization-wide or even enterprise-wide issues
- Remediating source code for developers in a prioritized fashion
With Moderne, platform and DevSecOps teams can work asynchronously to a CI/CD pipeline and do batch ingesting of LSTs to the Moderne platform to analyze and auto-remediate vulnerabilities at scale. They can offer recommendations to development teams about changes that should be committed—prioritizing by the level of criticality, strategic importance, or even “easy wins.”
The Moderne platform is able to do this because of its broad code purview—running recipes across potentially hundreds of millions of lines of code. As a side effect, these recipes are able to aggregate the structured data across a large set of code assets and produce what Moderne calls “data tables.” The data tables, expressed in tabular or columnar formats like CSV, Excel, Parquet, ORC, and so on, can be used to reason about code changes within and across organizations.
For example, consider this excerpt of a vulnerability data table in Figure 3 produced by running the Find and fix vulnerable dependencies recipe. This data shows every security vulnerability detected in direct and transitive dependencies for the ingested LSTs, as matched against the GitHub Advisory Database, including the CVE number, description, actual version of the dependency in use, the minimum fixed version, severity, and depth of dependency.
This data shows that you can close many critical severity vulnerabilities by bumping to the latest patch release of the minor release of Jackson.
This recipe also enables you to make immediate progress by upgrading the dependencies. We have a customer that runs this recipe once a week to keep code versions minimally upgraded as a general practice.
The data also enables you to see the path toward fixing the rest of the critical vulnerabilities that require major updates. And yes, there are recipes for that.
Why auto-remediation is the way forward for DevSecOps
There’s no longer a need for development teams to make daily decisions on whether to release software on time or improve code security and quality. They can do both by employing auto-remediation as part of a DevSecOps practice.
The Moderne platform offers the flexibility to automate code remediation in the continuous delivery path. But it also can work asynchronously toward larger organization goals, providing automation-driven pull requests or commits that become part of upcoming development sprints. Automation really just replaces manual work, and developers are still in control of reviewing and putting updates through their regular workflows.
Why is auto-remediation the way forward? For the answer, we can look at the results in the Moderne platform itself. Scanning tools will often provide advice for implementing a remediation, along with an estimated time each remediation will take (e.g., 2, 5 or 8 minutes per change). For the Common Status Analysis Issues recipe, Moderne uses the scanning tool time allotment (in this case from SonarQube) to show you the estimated hours saved for all the changes.
In this example of 154 repositories searched, it would have taken a development team 452 hours to complete all the changes manually (that’s about 11 work weeks!). Does anyone really think all of those changes would be made? With Moderne, the impossible becomes possible.
At the time the recipe run completes, the code changes are made to the LST (in this case, touching 94 repositories), and they are ready for review and submission back to the repo.
The Moderne platform does the remediation work for developers, automatically improving the security and quality of the source code. Teams can say YES we will fix these issues AND we will complete our feature development in time for release.
To try Moderne for yourself, use our public platform that ingests more than 300 million lines of open-source software ready for your transformation. You’ll be able to try out the full marketplace of recipes and see the time saved from auto-remediation. Then let’s talk about how Moderne can help you shift left in terms of security responsibilities without shifting the pain.
Also, you can contact our partner Zivra to learn about how their DevSecOps Advisory can help you modernize your security and development practices.
