E0308 mismatched types: perhaps two different versions of a crate are being used

[jo3bingham]

My team has a small project that depends on avro-rs(0.7.0) and schema_registry_converter(1.1.0)[which itself depends on avro-rs]. Recently, a team member cloned the repo for the first time, tried to compile, and was met with the E0308 error pointing at the use of avro_rs::types::Value; saying it expected a reference of that type but found a reference of the same type with the note: "perhaps two different versions of crate avro-rs are being used?"

I had them install and run cargo-tree to see what version of avro-rs their schema_registry_converter was using as I couldn't reproduce the issue and mine was using 0.7.0. They noted that it was using 0.9.0, and that changing our avro-rs dependency to 0.9.0 fixed the build issue (which makes sense). Curious as to why our dependencies differed, I checked crates.io and it said that schema_registry_converter was last updated 10 months ago so there's no way my teammate could have moved to a new version. However, I noticed in their Cargo.toml file that the avro-rs version is set to >= 0.6.0.

Correct me if I'm wrong, but my assumption is that, when a dependency is set like so, and there's no cached version, cargo will pull the latest version for compilation. I then tried changing my own avro-rs dependency from 0.7.0 to 0.9.0 and the build failed with the same issue, but I was able to resolve it by clearing the cached dependencies and rebuilding.

I guess my problem/issue/question is, what needs to be done so that we don't encounter this issue when avro-rs updates again? Obviously, we can just update our dependency, but it seems like a waste of time to either keep an eye on the avro-rs crate for updates, or wait for a build to fail. I believe we could clone schema_registry_converter and set the avro-rs version ourselves, but that seems like a hacky workaround?

Also, is there a reason that cargo doesn't take other dependencies into consideration during the build step? What I mean is, in this instance avro-rs gets pulled and compiled before schema_registry_converter; why does crate not see that instance of avro-rs while it's compiling schema_registry_converter then check the version and see if it's usable before pulling it's own dependency?

[Eh2406]

First off I am sorry for the pain this is causing you. I wish I had a ezy fix. We are working on a number of things to at least make this easier.

I had them install and run cargo-tree to see what ...

We recently included a new tree command in cargo itself (#8062), so in a few weeks it will no longer be a separate install. and by "we" I mean @ehuss.

However, I noticed in their Cargo.toml file that the avro-rs version is set to >= 0.6.0.

This is an odd decision for schema_registry_converter to make. It is saying that the schema_registry_converter will work with any breaking changes that avro-rs may make at any time in the future. I don't know how they are comfortable promising that. It is particularly painful as it is part of schema_registry_converter's public api, as demonstrated by your hitting E0308.

what needs to be done so that we don't encounter this issue when avro-rs updates again?

If we had RFC 1977 implemented then schema_registry_converter could tell cargo that its publicly re exporting its dependency on avro-rs. But it is held up on syntax and performance.

A common alternative if for schema_registry_converter to provide the dependency as a module, pub extern avro_rs (I think that is the syntax), so then you can get the compatible types with use schema_registry_converter::avro_rs, and not need the direct dependency.

Also, is there a reason that cargo doesn't take other dependencies into consideration during the build step?

Cargo makes all decisions about what the dependency graph looks like deterministically and before building starts. It does look at the lockfile to try to minimize the number of existing crates that will need to be rebuilt do to changes to Cargo.toml. Dependency resolution is complex (NP-Complete) and there are a lot of contradictory demands made on it. The users want "the latest version of everything" and "no rebuilding when adding dependencies" and "minimize the number of things built" and "easy use dependencies while mager updates are partially adopted by transitive dependencies" and ... We do our best.

[ehuss]

Yea, I'd echo what @Eh2406 said, and:

what needs to be done so that we don't encounter this issue when avro-rs updates again?

I would recommend not using >= dependencies, as that is the source of this problem. ^ tends to be safer (that is the default if there is no special symbol like 0.9.0). There are services like dependabot to keep things up-to-date.

why does crate not see that instance of avro-rs while it's compiling schema_registry_converter then check the version and see if it's usable before pulling it's own dependency?

As @Eh2406 mentioned, I think (not sure) that public/private dependencies (RFC 1977) might help with this, but it is not fully implemented.

[jo3bingham]

@Eh2406 @ehuss brilliant! Thank you for the quick responses.

This is an odd decision for schema_registry_converter to make.

I'll open an issue with them and, for now, go the route of cloning it myself so I can set the version because, as you mentioned, I don't want to get to a point where avro-rs pushes a breaking change for schema_registry_converter.

Since this isn't an actual bug with cargo I'll close this now. Thanks again!