[Epic]: Rosetta migration to standalone repo

[bizk]

Summary

The goal is to move Rosetta tool under cosmos-sdk/tools folder to a separated repo, in order to have a cleaner well structure maintenance and management of Rosetta efforts.

Problem Definition

While working on features and improvements over Rosetta. Our team have experienced some common issues related to dependencies and versioning due to the nested Rosetta project. In order to guarantee that the cosmos implementation of Rosetta always work and since it relies on Cosmos-sdk releases to work. We propose to detach it from the root repo.

Here are a few arguments pro decoupling Rosetta tool from cosmos-sdk

• Modularity and Decoupling: By separating the nested project into its own repository, it achieves better modularity and decoupling between the two projects. Enabling separate versioning, releases, and development processes. Changes made to one project will have minimal impact on the other, reducing the risk of introducing unexpected issues or dependencies it has been seen on some refactors.
• Dependency Management: Keeping the nested project in the same repository as the root project can lead to challenges in managing dependencies and versioning. When both projects share the same repository, they often end up sharing the same dependencies, making it difficult to have different versions or configurations for each project. The idea is to have a working Rosetta version per release which is always up to date and maintained.

Work Breakdown

In order to detach this project we need to rollback some changes so Rosetta can depend only on the latest cosmos-sdk release. Which at tis time is v0.47.x. See issue #16273

[tac0turtle]

we can fix this in the repo quite easily, just dont import rosetta into the sdk and then import a released version. Im not 100% convinced it needs to be moved yet.

The question i still have is will rosetta depend on the sdk or would it use grpc reflection to get all the needed data? you mentioned some plugin system but still unclear what is happening with rosetta. Mainly asking because there is always the chance that rosetta will fall on the sdk teams shoulder and I dont want to endup in a world where we delete lots of work to get it to its simplest state.

[bizk]

The fix was easy but it took us some time, we rolled back some changes so rosetta can depend only on latest cosmos release. The changes were made (probably by mistake) when doing some refactors. We would like to avoid that in the future.

Rosetta depends on Cosmos-sdk for the decoders and interfaces. We were working on the plugin functionability that allows users to extend Rosetta to use it on their zones (you can see we have some example for cosmos-hub and osmosis zones). Eventough the best solution would be to use gRPC reflection as you mentioned (and as hubl does), after trying to use hubl against multiple zones we came into the conclusion that we depend not only on other zones to use the latest release of Cosmos-sdk but to implement one service which is not mandatory to do so.

The idea behind this Epic is to make Rosetta easier to work with and maintain, following the release cycle and making sure that each release of cosmos will have a working release of Rosetta. About the new features, the objective is to maka rosetta cosmos-sdk implementation extendable to other zones

[julienrbrt]

The thing is rosetta is getting tagged from release branches, the latest version of rosetta has been tagged from release/v0.47.x (https://pkg.go.dev/cosmossdk.io/tools/rosetta?tab=versions / https://github.com/cosmos/cosmos-sdk/blob/tools/rosetta/v0.2.1/tools/rosetta/go.mod).
It is expected that the version from main do not always works, as indeed, it is as main is using development versions.
This actually the case for any go modules that directly depend on the SDK, they need to be tagged from release branches.
Modules not depending on the SDK (math, core, collections) are in other hand, released directly from main.

[alexanderbez]

we can fix this in the repo quite easily, just dont import rosetta into the sdk and then import a released version. Im not 100% convinced it needs to be moved yet.

Yes, this is technically true and will also allow you to get what you're looking for. By removing any usage of rosetta in the SDK and pinning the SDK version in rosetta's go.mod, this should allow you to get what you want.

[bizk]

You both have good points @julienrbrt @alexanderbez, if we work on fixes over Rosetta, our users wont be able to access them until we launch a new release of cosmos. Which in a way slows down adoption and forces people to create forks and so on.
If Rosetta consumes cosmos-sdk, isn't a better idea to have it as decoupled as possible? from our experience working with the latest changes when these are not polished / fininshed yet doesn't bring much value to the tool. We believe that following the release cycle is a cleaner way to work, specially for these cases where the tool consumes the project as any other user would do

[alexanderbez]

Yes, I'm not against spinning Rosetta out! I'm actually in favor of it. Just agreeing with @tac0turtle is all. But yah, you have my support on that.

[bizk]

Thank you @alexanderbez ! Hi @tac0turtle ! what do you think? what can we do from our side to move this forward?

[tac0turtle]

it seems there is a proposal to use plugins, has reflection been explored in order to remove the dependence on cosmos-sdk.

Looking at the plugin system, would be interested in understanding why hashicorp plugins wasnt considered since we use this in the sdk and it would be good to align all things if the plugin system is picked.

Mentioned in the call, is there any chance a quick adr/rfc can be written?

[bizk]

Hi @tac0turtle thanks for sharing your concerns!

Yes, we considered using the same reflection mechanism as HUBL tool does, since the different zones need to implement a specific (optional) service in order for us to register their interfaces + they need to upgrade to v0.47.x at least (most of the ones we checked are running on v0.45.x) we decided for now to run the plugin approach. We want to migrate to reflection eventually, but we believe it might take some time until it is worth it.

I have reviewed the hashicorp plugins system, our approach is the same: Compile your code, then register it (by adding it to the folder). By the docs and code I have been reading, I believe hashicorp system is designed with another purpose in mind (I might be wrong), thus I don't think registering interfaces would be a good use case for it.
You can take a look at what we are experimenting with in our fork we are quite happy with the results we are getting, since it is clean and easy to use, we were able to onboard cosmos-hub and osmosis in a few hours.

Let us write the adr/rfc and share it with the team so we can discuss it 🚀

[bizk]

Hey @tac0turtle, do you want the ADR about the epic, for the plugin system or for both things?