We’ve all dealt with CI gates that provide no help. You are trying to ship something, and the business screams at you that it’s late, and then CI breaks because security found a new vulnerability in existing code, and now you have to deal with it.
For open source projects, you may run into similar issues—where there are certain standards or conventions that the project requires that aren’t immediately obvious to someone wanting to contribute to the project. This can be especially irksome if you need to wait weeks for a review only to be told that you have to go rename a few methods and then you have to wait for a review again.
We want the contribution experience to OpenRewrite to go much more smoothly—where people don’t need to wait for us to review the code to address common issues that can be automatically fixed. To address this concern, we’ve developed a bot that will run OpenRewrite recipes against PRs and annotate the PRs with the results as comments. Each of these comments has a clear suggestion that you can accept with one click. Any OpenRewrite recipe can be included and run—from enforcing stylistic consistency and best practices to security remediations.
In this blog post, we’ll walk through what inspired us to create this bot, how we arrived at this solution, and how you might add a similar solution to your own organization.
What inspired us to automate PR annotations
At Moderne, we are dedicated to supporting the wider OpenRewrite community. This primarily comes in two parts: we maintain the core OpenRewrite project and we review and help distribute recipes created by the wider community.
We are incredibly fortunate to have so many people contributing recipes and helping to build out the scope of the OpenRewrite auto-refactoring ecosystem. That being said, as a small team, it can be incredibly taxing to review the substantial amount of PRs coming in so that the changes can get out to our users. Because of that, we often won’t get to PRs for a few days at least.
Furthermore, when we do review PRs, we don’t want to come back with a ton of red x’s—especially when many of the things we’d call out are fairly minor stylistic conventions that help us keep projects standardized and clean.
By automating certain checks, we can ensure that the people who are taking their time to contribute to the project don’t feel like their time is being wasted. In addition, when we get around to reviewing the PR, the automatic issues should be resolved, so we can focus on ensuring the core code makes sense.
Figuring out what automated checks are needed
While automation can help a project, if you don’t understand what the core, common issues are, you risk wasting your time developing automations that help nobody.
For us, we reviewed a lot of pull requests before we ever started automating anything. In doing so, we began to recognize repeated patterns that we wanted to see handled differently. For instance, some of the key issues we saw were:
People naming variables or methods that clashed with the rest of the project
- People including `System.out.println` in the submitted code
- People forgetting to add an `Override` annotation to their methods
While these issues are fairly small in scope, they do help us maintain consistency across the code base and prevent future issues.
If you’re curious to see our entire list of rules we’ve decided to automate, check out the best practices recipes in the rewrite-recommendations repository or in Moderne.
Initial code cleanup using the Moderne Platform
Once we had our list of issues we wanted to automate, our first task was to ensure that all of our projects followed those conventions. If we didn’t do this, PRs might have irrelevant suggestions about code that people did not directly contribute. It’s also a good way of double-checking that applying these automations won’t negatively impact the project or make our own PRs worse.
Of course there’s no better way to roll out such changes at scale than with Moderne. Using Moderne, we were able to quickly update not only projects we maintained, but we were also able to submit PRs to repositories that are maintained by the community that could also benefit from these recipes.
Using GitHub Actions to automatically update PRs
With all of the repositories updated according to our new standards, the next step was figuring out how to configure PRs to run our automations and provide suggestions that people can apply with one-click.
This can be broken up into two main pieces:
- A GitHub action that detects a pull request has been opened and then runs the desired recipes against the code.
- A GitHub action that takes the results of the above run and comments back to the PR with the suggested changes.
To help ensure that we can scale this across many repositories easily, we created a common workflow for running the recipes—separate from the detection of a PR being submitted. Then, in each repository we wanted to support, we added a workflow that detects a PR was submitted and then calls out to our shared workflow. By doing it this way, if we wanted to change the core recipe run workflow, we only need to update one file rather than updating every repository that runs our automations.
We then did the same thing for the piece that comments suggestions back. We created a common workflow that downloads the patch file generated from the previous step, applies the changes, and comments those back to the PR. Then, in each repository we wanted to support, we added another workflow that calls out to the common one to add the comments to that specific PR.
In addition to the above reasons, there is another key reason to separate these: security. The first step handles potentially untrusted code and, therefore, needs to be treated a bit differently, whereas the second step has a token that’s allowed to write comments on the PR. This blog post goes into more depth about how to safely run automations on untrusted code, as we’ve done here.
Seeing automated PR annotations in action
With our workflows done, we were finally ready to release it to the world. You can see it in action in a PR where a lovely member of our community submitted some useful changes to the rewrite-migrate-java repository.
Upon submitting the PR, our PR bot immediately ran the best practice recipe against their code and found a few issues. Rather than having to rely on us pointing these out ourselves when we got around to reviewing the PR, the bot was able to immediately point them out AND provide the solution for them (so they could just press “Commit suggestion” and resolve the issue). By the time we looked at the PR, all of these issues were resolved!
Below you can see two examples of this in action:
Using the Moderne CLI to annotate PRs
The above works great if you have a small number of highly homogeneous projects. But in a larger organization you’re more likely to have all kinds of different projects that use different languages and different build tools. You would need to create different recipes for each of these.
At Moderne, we’ve developed the Moderne CLI to help abstract away any such differences and give you a common interface to run recipes across projects, no matter what they require to build. If you want to use the Moderne CLI to automate PR checks, we’ve developed some sample receive and comment GitHub actions that do just that. In Figure 4, you can see a flow diagram that demonstrates all of the steps starting from a PR being opened and ending with comments being published back to said PR.
Please note that you can use the Moderne CLI to annotate PRs in any type of repository or project. In addition, both the Moderne SaaS Platform and Moderne CLI are available for free use on open source code.
Get the recipe for streamlined CI quality gates
Our goal with OpenRewrite has been to provide a helpful and welcoming community for contributors, and we see this automation of PR annotations to be an extension of that. You can hear the whole story at our Community Office Hours:
Also, please join the OpenRewrite Slack or Discord channels to get involved in building auto-refactoring automation for your organization.
And if you want to learn more about automating mass-scale code changes in your organization, contact us at Moderne.
