Ensure `miden-base` versions match in RPC header check

[tomyrd]

There is also a different but related problem that we can also have where miden-node and miden-client use a different miden-base revision. Client integration tests catch this (so we push a PR to the node when we need to), but there have been several instances where users themselves maybe update the client but not the local node they are running, and so they may get a more cryptic error. The problem may even look like a client problem at first sight because it's at the execution phase, for example, but in reality the problem is the incompatibility with node-incoming data and the client's data. So I wonder if we could do something for this case as well.

Originally posted by @igamigo in #804

Apart from the node<->client crate version check, there's another more niche case where the crate versions match but the miden-base version is different in each instance. This specific error shows up as an AdviceMapKeyNotFound error in the client so it's not always obvious that the issue is a version mismatch.
This error case is more common when developing but as Ignacio mentioned above, it may happen to users in specific instances.

[Mirko-von-Leipzig]

I recall we did discuss this back then but we didn't come to any conclusions.

I'm not sure we can fully guard against this on the node without requiring absolute commit SHA matches. To summarize my understanding:

• For formal releases, this would represent a bug in the semver of either base, node or client and should be equivalent to the RPC versioning.
• This can also happen during development when the node and client use incompatible (but also in-development) base commits.

I don't think there is a practical way of guarding against. One cannot use the base version - because its not a released version, but rather a commit. We can match on commits but then we're effectively in lockstep development between the node and client. And even then users on different branches could experience this.

TL;DR - I don't believe there is a pragmatic solution to this while we are both developing on a base that is also developing. The only safe time to use the node or client by a user is from a released version.

[bobbinth]

Here is my understanding (which I think aligns with what @Mirko-von-Leipzig wrote above):

• This shouldn't be a problem for released versions (e.g., on the testnet) unless we make some kind of a semver bug (e.g., release a new patch version of miden-base or other upstream dependencies which introduces a breaking change).
• This could be a problem for in-development version (i.e., on the next branch) because we frequently use git dependencies and semver checks are pretty much useless here.

The only potential way to address the second point I can think of is to have something akin to nightly releases. If the user uses the same nightly version of base/node/client then, they should expect things to work - and if the versions diverge we'd be able to detect this and tell the user.

In our case, the pre-releases don't need to be nightly - e.g., we can do them weekly. But this could be pretty time-consuming - so, I'm not sure we are at the point yet when we want to do this.

[igamigo]

We can match on commits but then we're effectively in lockstep development between the node and client

I'm not sure that this is a problem (this is sort of already the case). Because it's an in-development version, it's also expected that it won't be always stable. The main problem here is making sure the user gets a valid, useful error instead of a random one that depends on what the difference between miden-base versions is: for instance, you could get a deserialization error or the more cryptic AdviceMapKeyNotFound one, and none of those is particularly useful to the user (although some are more telling for sure). This does not have to necessarily be an error, it could be a warning of some sort.

Based on the number of times this has surfaced, I also think that it's fairly easy to get into a situation where you are testing with the updated client on next but disregard updating the node and the versions drift. But I agree there might not be a simple but great solution here. Some other alternatives would involve just documenting these scenarios better and providing a valid node from the miden-client repo itself that will always be compatible (basically the one we use for the CI, where we decide the version), but these are not great either.

[Mirko-von-Leipzig]

@drahnr could you take a look and see if you can think of something sensible? The main concern is that we don't really want to restrict formal releases to specific miden-base SHA matches. But maybe there is something one can do if the miden-base is a git branch etc instead of a version'ed release?

[drahnr]

Opened #1065