Poor handling of extension unavailability for nightly releases

[joepie91]

Sometimes extensions are not available for a nightly release, such as regularly happens with rls. When this occurs, a package specification like this:

(nixpkgs.rustChannels.nightly.rust.override {
	extensions = [
		"rust-src"
		"rls-preview"
	];
})

... will outright fail to build, and break the entire rebuild of a system, with an error like this:

error: attribute 'xz_url' missing, at /nix/store/kcdh3m23dy3vxzlrwj3kwjg02616kgbd-source/rust-overlay.nix:100:31

Ideally it would then just fall back to the most recent nightly release that does have the specified extensions, so as to not break rebuilds nor make extensions magically vanish.

Perhaps the nightly Rust attribute implementation could be changed to do that, so that one doesn't have to manually track down a nightly version that has what they need?

[nbp]

How does rustup behave in such cases, if the latest nightly does not provide the component that is supposed to be installed?

[joepie91]

I don't have any idea personally since I don't use rustup, but even if rustup refuses to install an unsatisfiable set of extensions, I don't think that that behaviour should be mirrored here.

Whereas rustup is typically only invoked when the user wants to explicitly change something in their Rust environment, that is not true for this overlay; the overlay package gets implicitly reevaluated every time the system is rebuilt. In my opinion it would therefore be reasonable to assume that if a newer version can't be installed under the specified constraints, the user would want the last satisfiable version instead.

[Shados]

@nbp rustup update either succeeds for all components, or doesn't change anything and prints an error listing the unavailable components for the newer version.

[nbp]

@Shados Thanks for the information.

So the current behavior is the one expected by rustup, if you were to do a fresh install all extensions without any prior installations. Thus, this overlay is actually correct in behaving this way by default.

So I think we should open an issue against the manifest which are being produced, to include a reference to the last manifest which satisfy all extensions, if it is not available in the current manifest.

On the other hand, if you want to implement a way to walk back the past 15 versions of nightly to satisfy an extension request, this sounds fine with me. However, this should not be the default.

[evanjs]

I've been trying to figure out a way to circumvent this, but tryEval isn't working, and I can't figure out how to check if xz_url is available per extension.

At the very least, I'd love to see stuff like xz_url exposed (if it isn't already) so we could at least hack our own solutions together.

As for me, for example, I'm trying to make a list of extensions in terms of precedence, update nightly, and go down the list if the preferred options fail.

e.g.
Try to get all extensions
If we can't get all of them, try the utilities (e.g. rustfmt, analysis, clippy)
If we can't get those, try std+src, etc

Case in point, this seems counter to the expected behavior even for rustup so I'd probably be hacking it in, anyway.

[evanjs]

Might these new rustup features propagate into this overlay somehow?

[rizary]

@evanjs I encouter with this issue when trying to use this overlay, is there any way around? Right now I have to use rustChannelOf from another repo I found. The downside is I don't know where to get the attribute date in the rustChannelOf set, so I'm not sure whether I am using the latest nightly or not.

[evanjs]

I recently had some time to take a look at this again.
I decided to go for an "optional features" approach, and it seems to work nicely (I think??)

The relevant commit/branch is evanjs@29ebd0c

Here is an example of how I am using it

let
  moz_overlay = import ((import <nixpkgs> {}).fetchFromGitHub {
    owner = "evanjs";
    repo = "nixpkgs-mozilla";
    rev = "29ebd0c1f68264ef3c79b9324e3908d717c8a5f9";
    sha256 = "0rl975v0np6jwg2zzmrl1hcrmfix4c985ild80dxc3ychbqgsmsp";
  });
  pkgs = import <nixpkgs> { overlays = [ moz_overlay ]; };
in
  with pkgs;
  pkgs.mkShell {
    buildInputs = with pkgs.gnome3; [
      (pkgs.rustChannels.stable.rust.override {
        extensions = [ "rust-src" "rust-std" ];
        optionalExtensions = ["rustfmt-preview" "clippy-preview" "llvm-tools-preview" "rust-analysis"];
      })
      gobject-introspection
      gtk
      glib
      gdk_pixbuf
      at-spi2-core
      libconfig
      gstreamer
      pkgs.openssl
      pkgs.pkgconfig
      (pkgs.vscode-with-extensions.override {
        vscodeExtensions = with pkgs.vscode-extensions; [
          llvm-org.lldb-vscode
          matklad.rust-analyzer
        ];
      })
    ];
  }

With this optional features approach, I always get the latest nightly, as long as the extensions specified in extensions are available. If any optionalExtensions are not found, they are simply ignored, and I still get the toolchain and any remaining optionalExtensions, as well as all extensions.

@rizary