Pass arguments from the command line into a single configurable recipe

[commonquail]

We can already run OpenRewrite via the Maven plugin without modifying files up front:

mvn -U org.openrewrite.maven:rewrite-maven-plugin:run \
  -Drewrite.activeRecipes=org.openrewrite.java.RemoveUnusedImports

but sadly only for recipes that require no configuration. Recipes that do require configuration have no (canonical) inline invocation form, instead necessitating the creation of a verbose and complex file merely to communicate what is often trivial information to the recipe, and then you even still need the rewrite.activeRecipes user property from the example. This severely limits the practical utility of OpenRewrite. Instead of being a hypothetical arbitrary rename-refactor the tool is only really interesting for completely static transformations, or for dynamic transformations that are particularly complex or that have to run in batch mode.

What problem are you trying to solve?

Consider a trivial recipe with a structure comparable to

---
type: specs.openrewrite.org/v1beta/recipe
name: com.yourorg.ChangePackageExample
displayName: Rename package name example
recipeList:
  - org.openrewrite.java.ChangePackage:
      oldPackageName: com.yourorg.foo
      newPackageName: com.yourorg.bar

Suppose this needs to be run by many different individuals for different values of oldPackageName or newPackageName. For <reasons> we can distribute instructions for defining and executing a recipe but distributing a recipe itself is impractical; say, for example, that we cannot know literally every variation of package names, or that the old package name can exist in multiple artifacts and should be turned into numerous divergent package names.

The YAML configuration file works well with distribution when distribution is already a solved problem, but distribution is a difficult problem to solve and cumbersome even when already solved. It requires a distribution channel, which you may not already have, and if you do have one, using it may necessitate a disproportionate amount of bureaucracy namely if you're subject to auditing regulations. Even if distribution is feasible, some recipes are impossible to distribute in practice, like the examples from the previous paragraph.

But I don't see why we couldn't "just" pass the package names as user properties like we can with rewrite.activeRecipes.

Describe the solution you'd like

Imagine I could run something like

mvn -U org.openrewrite.maven:rewrite-maven-plugin:run \
  -Drewrite.activeRecipes=org.openrewrite.java.ChangePackage \
  -Drewrite.ChangePackage.oldPackageName=com.yourorg.foo \
  -Drewrite.ChangePackage.newPackageName=com.yourorg.bar

Generalizing, -Drewrite.RecipeSimpleName.optionName=value.

This is a command I can email blast to lots of individuals, each of whom can easily adjust the invocation as necessary with no further effort.

The pattern I propose means you can run each distinct recipe only once per invocation but is acceptable to me. Repeating the command for variations in configuration option values is trivial, and if you need to do that a lot you're probably in a situation where the overhead of the configuration file approach is amortized anyway. An n-ary invocation form is theoretically possible, it just doesn't seem worth the implementation complexity or the impact to invocation complexity.

I don't care about the exact format of the user property. It only needs to be reasonably resistant to collision and should feature the option name verbatim for clarity.

Have you considered any alternatives or workarounds?

On multiple occasions I've discounted OpenRewrite in favour of low level tools like sed and mv and Python on account of this limitation. I've thought about making tools to generate specialized OpenRewrite configuration files but that has not been worth the effort so far.

[timtebeek]

Hi @commonquail ; thanks for explaining clearly what you're after in your use of the tool. Did you already happen to come across this entry into our FAQ?

Is it possible to pass arguments to a recipe from the command line? I want to programmatically configure complex recipes.
Not right now. This is a particularly difficult problem to address for a couple of reasons:

• Some recipes can be composed of other recipes which could then include other recipes composed of other recipes and so on.
• Some recipes can be used multiple times in one recipe with different parameters such as in this example.

There is an open issue for this request that you can +1 or provide feedback on.

You might be interested to have a look at the Moderne CLI: There we have a mod run command that's able to take in --recipe-option for quick runs against serialized LSTs: https://docs.moderne.io/user-documentation/moderne-cli/cli-reference#mod-run
The CLI also integrates with our Moderne IntelliJ plugin, such that you can search and run across repositories more easily. There's a link to get started at the end of the blog.

It sounds like the CLI might align well with the experience you're after, while supporting more than just Java Maven and Gradle projects. For the OSS Maven and Gradle plugins such an approach might not work as well, given the time it takes to build a model versus running just a single recipe. With the CLI there's no such delays for multiple quick one-off recipe runs.

Of course there's always the option of wrapping your recipes up into an internal recipe library, based on our rewrite-recipe-starter. Most companies will have the infrastructure to download a jar from Artifactory or Nexus, using -Drewrite.recipeArtifactCoordinates=g:a:v. That won't solve all the use cases you've listed above, but could still help.

[commonquail]

Did you already happen to come across this entry into our FAQ?

Oops, no. I figured something like that would have been referenced in the documentation I linked.

I appreciate why composite recipes are Difficult™. It's why I would have personally just ruled them out of scope.

It sounds like the CLI might align well with the experience you're after

Not really, no. The appeal of Maven (Gradle) is that its distribution can be presumed. The whole point is that I can't control remote file systems -- delegating to another tool isn't any easier than copying a YAML file, and in fact can be a great deal more difficult for executables.

Of course there's always the option of wrapping your recipes up into an internal recipe library

Yes, this is the difficulty I alluded to. It's a mechanism, sure, but it is incapable of addressing the case I presented.

[timtebeek]

Did you already happen to come across this entry into our FAQ?

Oops, no. I figured something like that would have been referenced in the documentation I linked.

Thanks for the hint! I've added a link to the FAQ to the page you linked above.

I appreciate why composite recipes are Difficult™. It's why I would have personally just ruled them out of scope.

Good that we agree on the the difficulties with composite recipes. The challenge then is to provide those additional options only to standalone recipe runs, and educate users about the difference and limitations. Add to that the limitations with providing good user feedback in a Maven plugin, and the fact that there's no immediate parallel to command-line only runs for our friends using Gradle. Taken altogether that made us choose for now not to add these options to the Maven plugin, but focus on a single user experience we can more easily control with the CLI.

It sounds like the CLI might align well with the experience you're after

Not really, no. The appeal of Maven (Gradle) is that its distribution can be presumed. The whole point is that I can't control remote file systems -- delegating to another tool isn't any easier than copying a YAML file, and in fact can be a great deal more difficult for executables.

Understandable that folks are more likely to have Maven installed as opposed to a new tool.

Have you thought of going the opposite direction? Not send out the commands for them to run, but the results of those commands in a PR? That's an approach that we see working well with users of Moderne, where a centralized platform team can help teams get up to speed with recipe runs, sending out the resulting PRs for review in the teams. Just as a way of looking how you can tackle the problems you're facing with existing tools and workflows, as opposed to trying to fit a particular usage pattern.

Of course there's always the option of wrapping your recipes up into an internal recipe library

Yes, this is the difficulty I alluded to. It's a mechanism, sure, but it is incapable of addressing the case I presented.

That might indeed be the case; it's always difficult to visualize a simplified and redacted use case, and I can understand that you can't share everything you're hoping to use these tools for. I can only then show you the pattenrs that we have seen work well, without knowing if those would fit exactly what you're after.

I hope you're still getting something from this discussion though, and we remain open to suggestions on how to improve.

[timtebeek]

We've discussed this some more internally @commonquail ; if you're up for it we'd welcome a contribution to take in command line arguments and pass them into a single (non-composite) configurable recipe. We could then incubate this for a while and see how that works out in practice.

I've moved the issue as this would be unique to the Maven plugin, as Gradle already requires an init.gradle that necessitates handling local files, which can then be extended to cover arguments there. We don't expect to get to implementing this ourselves any time soon.

[commonquail]

Have you thought of going the opposite direction? Not send out the commands for them to run, but the results of those commands in a PR?

Indeed. Working on behalf of people is not an option either, for bureaucratic or practical reasons or both. I mean, it's all a bit inefficient and error prone, to be sure, but these are the constraints.

That might indeed be the case; it's always difficult to visualize a simplified and redacted use case, and I can understand that you can't share everything you're hoping to use these tools for.

I didn't mean to sound secretive there, it's just that if the recipe's behaviour is necessarily contextual then it's just not possible to centralize the recipe (short of creating n concretizations of the same template recipe but in practice that's less feasible than written instructions).

we'd welcome a contribution to take in command line arguments and pass them into a single (non-composite) configurable recipe.

Cool! I can't make any commitments but it's nice to know you'd be willing to evaluate the functionality.

I've moved the issue as this would be unique to the Maven plugin

Thanks. I personally wouldn't have been able to help with Gradle in any case.

[marvin-froeder]

Any updates on this?

I'm on a somewhat airgap environment, where I can reach maven central (via proxy) but no way I can use moderne-cli as it is too reliant on external app

[timtebeek]

Just to be clear: the Moderne CLI has no reliance on any external app; it can work fully locally if you want it to, without any connection to the Moderne platform or DX. We'll use the exact same configuration as Maven does when connecting through a proxy to Maven Central. Welcome to give the CLI a try.

Other than that the offer from above still stands: if anyone wants to contribute similar argument handling for standalone recipes to the rewrite-maven-plugin, then we'll review and maintain that going forward.

[marvin-froeder]

Interesting, cause I couldn't move forward with the client without running

mod config moderne edit https://app.moderne.io --token mat-YOUR_TOKEN_HERE

but that gave me an error due to airgap env

[timtebeek]

You'll likely want to use mod config license edit YOUR_TOKEN instead, as described here:
https://docs.moderne.io/user-documentation/moderne-cli/getting-started/moderne-cli-license#everyone-else-including-multi-tenant-moderne-customers

In short we have various ways for folks to run the Moderne CLI, and optionally hook that up to a in-house or hosted Moderne instance. You've followed the instructions for folks that have their private instance of Moderne, whereas you'll want to follow the flow "for everyone else" as outlined in the docs.

Hope that helps you get started for now! For other folks: you can get a license through this form.

[thombergs]

I'm very much interested in this as well, as we're currently trying to leverage the Maven plugin to (automatically) execute arbitrary recipes from the command line without having to create rewrite.yml files on the fly.

I'll take a look at the code next week to see if I can contribute this and may come back with questions here :)