cargo update should not highlight non-semver-compatible "(latest: v0.23.5)" for indirect dependencies

[djc]

Problem

I just ran my weekly cargo update against the workspace at work, and it highlighted this change:

    Updating rustls v0.21.11 -> v0.21.12 (latest: v0.23.5)

This workspace does not have a direct dependency on rustls. As such, I would suggest this information isn't actionable enough to merit a mention in the non-verbose cargo update output (it would definitely fit in with the verbose output).

Steps

No response

Possible Solution(s)

No response

Notes

No response

Version

cargo 1.78.0 (54d8815d0 2024-03-26)
release: 1.78.0
commit-hash: 54d8815d04fa3816edc207bbc4dd36bf18014dbc
commit-date: 2024-03-26
host: aarch64-apple-darwin
libgit2: 1.7.2 (sys:0.18.2 vendored)
libcurl: 8.4.0 (sys:0.4.72+curl-8.6.0 system ssl:(SecureTransport) LibreSSL/3.3.6)
ssl: OpenSSL 1.1.1w  11 Sep 2023
os: Mac OS 14.4.1 [64-bit]

[heisen-li]

@rustbot label +Command-update +A-console-output

[weihanglo]

Thanks for the suggestion!

Agree it is not actionable locally and quite verbose, especially when you have only few direct dependencies and that's very noisy. However, I see it as an opportunity to encourage user to collaborate with upstreams. Putting in behind --verbose would reduce the change of it. It is also a more consistent console output that an update always comes with latest version information, but not some are behind --verbose some are not.

One similar work we've done is future-compatible warnings. Cargo indeed puts them behind cargo report, though I am not sure if they should have the same behavior. Should also weigh on the complexity added and the verbosity it reduces.

[djc]

Agree it is not actionable locally and quite verbose, especially when you have only few direct dependencies and that's very noisy. (..) Putting in behind --verbose would reduce the change of it. It is also a more consistent console output that an update always comes with latest version information, but not some are behind --verbose some are not.

This seems like bad reasoning, since cargo update --verbose reveals a bunch more crates that similarly have semver-incompatible versions (but somehow these are not being shown without --verbose):

Unchanged hyper-rustls v0.24.2 (latest: v0.27.1)

I suppose that the (latest ..) notice for rustls is only shown because there was a semver-compatible bump, but IMO that's not a good reason to also show the (latest ..) notice for non-direct semver-incompatible versions.

If anything, I would be happy if you showed Unchanged semver-incompatible bumps for my workspace's direct dependencies, but it currently doesn't -- so that also feels inconsistent.

To summarize:

• I think Updating notices should not show (latest ..) if the latest version is semver-incompatible with the selected version (because that doesn't fit in with the scope of cargo update)
• Maybe the color should change for (latest ..) notices advertising a version that is semver-incompatible with the selected version, making it green instead?
• Even for Unchanged notices from cargo update --verbose it's not obvious to me that it makes sense to show (latest ..) notices for semver-incompatible versions, since that's historically outside of the scope of update
• On the other hand, it would be great to start extending cargo update to help users address semver-incompatible bumps, and one good way to go about that would be to start noting available semver-incompatible updates for direct dependencies using a (latest ..) notice independent from if the dependency got updated

However, I see it as an opportunity to encourage user to collaborate with upstreams.

(I happen to do a lot of this, trying to push the ecosystem forward -- but I think I'm in a small minority, and anyway it's still a different level of actionable/effort to effect changes like this, so any production user will have to timebox these efforts.)

(cc @epage who's done a lot of work in this area.)

[epage]

Sorry for the delay, I've been doing some focused time trying to wrap things up before I get to my 100+ github notifications between MSRV, 2024 Edition, and RustNL (wish timing worked to talk to you in person about this).

This also caused confusion in #13873 (comment)

This was previously discussed in #13372. The idea behind this comes from

• A desire to help users know about dependencies held back generally (cargo update should print warnings for incompatible candidates #7167, Make cargo outdated part of cargo itself #4309)
  • MSRV and semver are two ways dependencies can be held back and we should acknowledge and handle both, as well as other reasons (missing feature, non-semver bounds, etc)
• On-going work to integrate cargo upgrade into cargo update (A way for users to bulk upgrade across incompatible versions #12425)
• The state of your entire dependency is relevant even if you can't directly act on indirect dependencies
• This is the more trivial solution so starting there and collecting feedback (which we did in the PR itself and with the call for testing for MSRV more generally) is the better approach

If anything, I would be happy if you showed Unchanged semver-incompatible bumps for my workspace's direct dependencies, but it currently doesn't -- so that also feels inconsistent.

I'd have to check the implementation and issues, but I have a feeling this runs counter to the feedback you gave for cargo upgrade.

CC @jonhoo as you mentioned in the release changes stream your excitement for reporting changes and want to make sure we capture multiple perspectives

[djc]

If anything, I would be happy if you showed Unchanged semver-incompatible bumps for my workspace's direct dependencies, but it currently doesn't -- so that also feels inconsistent.

I'd have to check the implementation and issues, but I have a feeling this runs counter to the feedback you gave for cargo upgrade.

I'm curious what feedback you're thinking of exactly!

I think giving this (latest: ..) feedback during cargo update is excellent and, if implemented well (consistently), takes a lot of the pressure off of cargo upgrade IMO -- if cargo upgrade will reliably tell me when my direct dependencies are outdated, actually upgrading them is the easy part. So I think there is a lot of potential here. What I'm unhappy about is the random nature (only dependencies for a semver-compatible update is available show whether there is a newer incompatible version available), which might make sense for the MSRV scenario specifically but doesn't make sense when the current workspace does not specify an MSRV target.

[epage]

With cargo upgrade, didn't you push back on a lot of our output unrelated to the actual upgrade, having us put it behind --verbose? Showing unchanged direct dependencies (that could be updated) runs counter to that.

What I'm unhappy about is the random nature (only dependencies for a semver-compatible update is available show whether there is a newer incompatible version available), which might make sense for the MSRV scenario specifically but doesn't make sense when the current workspace does not specify an MSRV target.

changed dependencies are shown, regardless of the nature of the change (cargo upgrade like functionality is my priority after MSRV/cargo script).

Why doesn't this make sense when an MSRV isn't set? Cargo is preventing an upgrade and we want the user to know. Right now, resolving some of those upgrades is more manual (edit version requirements, bug dependencies) but we will be adding support for a flag to help with the version requirements.

[djc]

With cargo upgrade, didn't you push back on a lot of our output unrelated to the actual upgrade, having us put it behind --verbose? Showing unchanged direct dependencies (that could be updated) runs counter to that.

IIRC the output in cargo upgrade I pushed back on was when it displayed a large table of dependencies many of which had no available updates at all, or only had semver-compatible updates where I was only expressing an interest in incompatible updates -- there was a bunch of inactionable output. In this case I'm arguing both that there is output that is inactionable and that there's missing output that would be actionable.

What I'm unhappy about is the random nature (only dependencies for a semver-compatible update is available show whether there is a newer incompatible version available), which might make sense for the MSRV scenario specifically but doesn't make sense when the current workspace does not specify an MSRV target.

changed dependencies are shown, regardless of the nature of the change (cargo upgrade like functionality is my priority after MSRV/cargo script).

Why doesn't this make sense when an MSRV isn't set? Cargo is preventing an upgrade and we want the user to know. Right now, resolving some of those upgrades is more manual (edit version requirements, bug dependencies) but we will be adding support for a flag to help with the version requirements.

Yes, I understand what it's doing and why it is the way it is. I'm arguing that whether the "(latest ..)" badge is shown should be independent of whether a dependency is changed (due to a semver-compatible upgrade/downgrade), because whether a semver-compatible update happened since the last time I ran cargo update or not is not meaningful signal -- it doesn't have any bearing on how actionable the "latest" hint is.

Including the plan for the MSRV-aware resolver, I think there are 3 different categories here:

• There is a semver-compatible newer version that was not selected because of a workspace MSRV restriction: this should be shown always (whether there was a semver-compatible update or not this time, because otherwise I would miss it if I'm stuck at the latest semver-compatible update when all newer versions are MSRV-incompatible).
• There is a semver-incompatible newer version in a direct dependency: I would argue that this should also always be shown, because it is likely actionable for the user, and again, whether a semver-compatible update happened since the last cargo update run is immaterial (it's likely that no semver-compatible updates will happen anymore now that a semver-incompatible newer version has been published).
• There is a semver-incompatible newer version in an indirect dependency: I would probably not show this by default (only for --verbose), since it is much less likely to be actionable for the user -- at the very least it will be different level of effort. But if we're showing this for dependencies that got a semver-compatible update during this run, I think we should also show it for dependencies that did not get an update during this run, because otherwise it's pretty inconsistent for users. (Also perhaps more likely to increase mob mentality in users pressuring dependencies to upgrade.)

[jonhoo]

Thanks for CCing me! Yeah, I think I agree with @djc above on the things that are important to highlight. In particular, I agree that dependencies that could not be updated to the latest version for a reason that is under the user's control (i.e., direct dependency or MSRV) should be highlighted independently of whether that dependency happened to be updated in this particular cargo update run. I also agree that showing non-actionable available updates should be placed behind --verbose.

I very much empathize with @weihanglo wanting to encourage upstream contribution, but as @djc points out there is a trade-off here where overloading users with hard-to-action information is likely to lead to worse overall outcomes (people ignoring warnings) rather than the virtuous effects we wish to see. I sadly don't have a great suggestion for how to highlight updates that are blocked indirectly. Personally, I also tend to run cargo outdated -R much more often than cargo outdated precisely because it's so hard to really do anything about the transitive blocks. I think that's a problem that's likely to need some materially different solution.