Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mix.install/1 on Linux tries to fetch a precompiled ...dll.tar.gz instead of ...so.tar.gz. #66

Open
chgeuer opened this issue Jan 6, 2024 · 3 comments

Comments

@chgeuer
Copy link

chgeuer commented Jan 6, 2024

This certainly isn't a bug, but my own lack of understanding of how Rustler and rustler_precompiled work, so please accept upfront my sincere apologies. Rustler and rustler_precompiled are pretty cool 😍.

I created an Elixir library ([:ex_windows_api_dataprotection](https://hex.pm/packages/ex_windows_api_dataprotection)), which calls into a Windows-specific API using Rustler. I use {:rustler_precompiled, "~> 0.7"} so that my users don't have to have Rust installed. Given that the API is a Windows-specific API, but the package might be installed by non-Windows users, the Rustler function returns the :only_available_on_windows. I took the :explorer .github/workflows files as a starting point to use rustler_precompiled.

When GitHub actions runs the release build, it creates perfectly fine Windows bits (ex_windows_api_dataprotection-v0.1.2-nif-2.15-x86_64-pc-windows-msvc.dll.tar.gz). When creating the Linux bits, it also correctly creates a libex_windows_api_dataprotection-v0.1.2-nif-2.15-x86_64-unknown-linux-gnu.so.tar.gz file (notice the .so.tar.gz in the filename).

However, when I run a simple script on a fresh Linux box:

Mix.install([{:ex_windows_api_dataprotection, "~> 0.1.2"}])

IO.inspect(Windows.API.DataProtection.wrap("Hello"))

Please note when you now run the same script, it'll work, because I did the workaround described below.

It tried to fetch the Linux bits from a .dll.tar.gz file (instead of .so.tar.gz):

15:50:47.621 [info] Attempt 3 failed with {:error, "couldn't fetch NIF from https://github.com/chgeuer/ex_windows_api_dataprotection/releases/download/v0.1.2/libex_windows_api_dataprotection-v0.1.2-nif-2.15-x86_64-unknown-linux-gnu.dll.tar.gz: {:ok, {{~c\"HTTP/1.1\", 404, ~c\"Not Found\"}, ...."}

15:50:48.647 [debug] Downloading NIF from https://github.com/chgeuer/ex_windows_api_dataprotection/releases/download/v0.1.2/libex_windows_api_dataprotection-v0.1.2-nif-2.15-x86_64-unknown-linux-gnu.dll.tar.gz

== Compilation error in file lib/internal/native.ex ==
** (RuntimeError) Error while downloading precompiled NIF: couldn't fetch NIF from https://github.com/chgeuer/ex_windows_api_dataprotection/releases/download/v0.1.2/libex_windows_api_dataprotection-v0.1.2-nif-2.15-x86_64-unknown-linux-gnu.dll.tar.gz: {:ok, {{~c"HTTP/1.1", 404, ~c"Not Found"}, "Not Found"}}.

You can force the project to build from scratch with:

    config :rustler_precompiled, :force_build, ex_windows_api_dataprotection: true

In order to force the build, you also need to add Rustler as a dependency in your `mix.exs`:

    {:rustler, ">= 0.0.0", optional: true}

I'm trying to understand why the mix deps.get or Mix.install/1 on Linux tries to fetch a .dll.tar.gz from the release share, instead of .so.tar.gz.

Workaround: I downloaded the existing .so... release, renamed to .dll... and manually added it to the release, which circumvents the problem for that specific release.

image-20240106170713037

After the workaround of manually adding the dll file to the release, you can see it being picked up by the sample script:

chgeuer@hp:~$ elixir wrap.exs

Resolving Hex dependencies...
Resolution completed in 0.115s

New:
  castore 1.0.5
  ex_windows_api_dataprotection 0.1.2
  rustler_precompiled 0.7.1
* Getting ex_windows_api_dataprotection (Hex package)
* Getting rustler_precompiled (Hex package)
* Getting castore (Hex package)
==> castore
Compiling 1 file (.ex)
Generated castore app
==> rustler_precompiled
Compiling 4 files (.ex)
Generated rustler_precompiled app
==> ex_windows_api_dataprotection
Compiling 2 files (.ex)

16:17:10.057 [debug] Downloading NIF from https://github.com/chgeuer/ex_windows_api_dataprotection/releases/download/v0.1.2/libex_windows_api_dataprotection-v0.1.2-nif-2.15-x86_64-unknown-linux-gnu.dll.tar.gz

16:17:10.973 [debug] NIF cached at /home/chgeuer/.cache/rustler_precompiled/precompiled_nifs/libex_windows_api_dataprotection-v0.1.2-nif-2.15-x86_64-unknown-linux-gnu.so.tar.gz and extracted to /home/chgeuer/.cache/mix/installs/elixir-1.16.0-erts-14.2.1/b77c538c8667667f7f4a42c185d7fffa/_build/dev/lib/ex_windows_api_dataprotection/priv/native/libex_windows_api_dataprotection-v0.1.2-nif-2.15-x86_64-unknown-linux-gnu.so
Generated ex_windows_api_dataprotection app

:only_available_on_windows
@philss
Copy link
Owner

philss commented Jan 6, 2024

@chgeuer thank you for the details!

I don't know yet what is going on, but I would like to know if you are publishing the package from a Windows machine. If so, it is probably a bug related to that. Thanks!

@chgeuer
Copy link
Author

chgeuer commented Jan 6, 2024

I indeed did the hex publish on a Windows machine.

@philss
Copy link
Owner

philss commented Jun 21, 2024

Sorry for the delay here! I found the problem, and it's very specific to your lib: we check for the string "windows" to determine if the extension is a DLL or a ".so". Since the lib is named "ex_windows_api", we always say it's a DLL.

I honestly don't know how to circumvent that problem :/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants