Unable to cargo search against panamax mirror

[alexjbuck]

When trying to use the cargo search command, I get this error:

$ cargo search <crate>
error: crates-io is replaced with non-remote-registry source registry `panamax`;
include `--registry crates-io` to use crates.io

I have set both environment variables:

RUSTUP_DIST_SERVER=http://127.0.0.1:8080"
RUSTUP_UPDATE_ROOT=http://127.0.0.1:8080/rustup"

This is my .cargo/config.toml:

[source.panamax]
registry = "http://127.0.0.1:8080/git/crates.io-index"
[source.panamax-sparse]
registry = "sparse+http://127.0.0.1:8080/index/"

[source.crates-io]
replace-with = "panamax"

And I edited mirror.toml to have the base_url be http://127.0.0.1:8080/crates.

Despite that, cargo search <crate> doesn't seem like it will search against the local panamax instance.

I couldn't find anything in the Readme or elsewhere that indicated whether or not cargo search should work with Panamax, but it seems like it would be a useful feature to have if its not yet supported.

[k3d3]

Hey,

Unfortunately cargo search does not work with the mirror.

Implementing it, while useful, would be a pretty big endeavour that I unfortunately don't have the bandwidth to take on right now.

I'd happily accept a PR for it though.

[alexjbuck]

Okay! I can't say I'll look into it soon, but good to know that it doesn't. At a minimum I might push a docs PR that states that cargo search doesn't work with this mirror.

Thanks for the super quick response!

[qtfkwk]

Found this issue the other day while looking for the same functionality... and have been working locally on a potential fix...

So far, I've made an as-of-yet unpublished crate with library and CLI that parses the .crate files in a panamax mirror's crates subdirectory. It takes about 18 seconds on my i9 / 64GB RAM machine to create a ~16 MB JSON cache file (wanted to use TOML for consistency but it doesn't work all that well with BTreeMap data; oh well at least it enables web and/or jq-like use). I'm using rayon to do in parallel the description extraction from the .crate files (which is a gzipped tar file).

The CLI enables loading from the cache file, performing a search, and then produces output similar to cargo search. (I say similar because right now, I'm just doing results with exact name match, name contains, and description contains, while cargo search does more advanced relevance queries. So maybe I'll figure how they do it and add it to this.)

This is what it looks like now:

And for comparison:

I'm also thinking to use this library from panamax itself... particularly to enable it to create and update the cache file during sync and hopefully more efficiently than this CLI tool, and to load it during serve and provide a warp route/handler so that cargo search would just work when configured against a panamax mirror... but I'm still early on that part.

When these are ready, I'll publish to crates.io and send a PR linked to this issue.

Also, please let me know if you have any other thoughts, concerns, etc.

[qtfkwk]

It takes about 18 seconds on my i9 / 64GB RAM machine to create a ~16 MB JSON cache file

Rewrote it with a better approach/design and have it down to a ~14 MB file in ~2 seconds, with update capability and no updates ~1 second.

[qtfkwk]

• Library: https://crates.io/crates/panamax-search-lib
• CLI: https://crates.io/crates/panamax-search
• Repository: https://github.com/qtfkwk/panamax-search

[qtfkwk]

Note that this still requires one to add --index http://panamax.internal/git/crates.io-index to the cargo search command and so far I've not been able to get it to work via any other change to ~/.cargo/config.toml or command line option. Please lmk if you have any ideas.

[qtfkwk]

Also note that the panamax-search CLI utility exists now, and runs directly against the local panamax mirror versus going through cargo and requiring a separate copy of that git repository. In all cases, except for searching a non-local panamax mirror, I think it is probably better.

[qtfkwk]

Configuration:

Set the correct listening panamax serve URL at:

• mirror.toml: crates.base_url
• crates.io-index/config.json