Support building Rust addons for Electron

[emk]

Thank you for building Neon! I've already got a small Rust-based addon in production at work, and it was dead simple to get up and running.

For my own amusement, I've been working on getting Neon to work an Electron app. This is trickier, both because of node-gyp issues, and because Neon always assumes it's building packages for the system code of Node.js. Here's how I worked around this.

This is for a top-level Electron app project with a native subdirectory directly in the main package, and not under node_modules.

1. Set up environment variables for node-gyp

Save the following file as .env.sh in the top level of the project:

# Source this with `. env.sh` before running `yarn` or `native/build.sh`.
# This sets up the configuration required by `node-gyp` and `neon` to build
# native modules that will successfully link against Electron instead of
# the system copy of NodeJS.

export NEON_NODE_ABI=50
export npm_config_target=1.4.12
export npm_config_arch=x64
export npm_config_target_arch=x64
export npm_config_disturl=https://atom.io/download/electron
export npm_config_runtime=electron
export npm_config_build_from_source=true

2. Run the Rust build manually with a fake $HOME

#!/bin/bash
#
# This script can be invoked from the root directory as `native/build.sh`.
# It will make sure we have the right Node.js headers to build a module for
# Electron and run the actual build manually.
#
# If this is giving weird resutls, try:
#
#     rm -rf node_modules native/target
#
# ...and try again.

set -euo pipefail

# Create a fake home directory under ~/.electron-gyp to cache things
# related to Electron builds.  We need to make symlinks to various Rust
# config directories if we want this to work.
mkdir -p ~/.electron-gyp
if [ ! -h ~/.electron-gyp/.cargo ]; then
    ln -s ~/.cargo ~/.electron-gyp/
fi
if [ ! -h ~/.electron-gyp/.multirust ]; then
    ln -s ~/.multirust ~/.electron-gyp/
fi

# Switch to the fake home directory.
HOME=~/.electron-gyp

# Make sure that our Electron Node headers are installed.
if [ ! -d ~/.node-gyp/iojs-$npm_config_target ]; then
    node_modules/.bin/node-gyp install
fi

# Run the actual build, and copy it to where node will find it.  Ideally
# we'd just call `neon build`, but that requires neon-bindings/neon#109 and
# neon-bindings/neon-cli#31 to have any chance of working.
case `uname -s` in
    Darwin)
        cd native
        cargo rustc --release -- -C link-args=-Wl,-undefined,dynamic_lookup
        cp target/release/libtestnative.dylib index.node
        ;;
    Linux)
        cd native
        cargo build --release
        cp target/release/libtestnative.so index.node
        ;;
    *)
        echo "Don't know how to build native extensions on this platform" 2>&1
        exit 1
esac

What would it take to support this reasonably in standard Neon?

The most important fixes would be something like:

• optional ABI override in neon build #28
• Add parameter to override Node module ABI version #31
• Take module version from node headers. neon#109

This would allow me to replace the ugly case statement with a normal neon build call, which would be a clear improvement.

I'm less sure that Neon should touch any of the node-gyp stuff. Some of this is handled by electron-rebuild for regular Node addons, and there might be some way to tie into that system. Or not.

Anyway, because everybody loves screenshots:

That shows the Neon-based addon being loaded into both the main and renderer process of Electron. It works! The full application is here.

[EdShaw]

node-gyp's devdir option allows you to point to a specific location to search for node headers. This avoids the shenanigans with the fake-$HOME. Not sure how well it plays with the rest of the npm ecosystem though - it potentially breaks other things.

[emk]

@EdShaw Could you point me at an example or web page with more information on devdir? Google is not cooperating at the moment. I found this diff but I haven't found the docs yet.

It looks like node-gyp takes a command-line flag, or one of XDG_CACHE_HOME, HOME or USERPROFILE. There's also some hint that maybe this can be passed through npm. So maybe export npm_config_devdir=~/.electron-gyp might do something?

Electron + Neon + Rust + a modern JS stack is a nice dev environment. I'd love to see this be easier to set up.

[EdShaw]

Your links are what I referenced. It's a fairly recent addition (3 months); it looks like the changes haven't made it into a release yet. :( Make me wonder how I got it working...

I'm trying to update node-gyp to see if it works...

I see in your script that you call node-gyp directly, I suspect only --devdir will work in that case (since I don't think it picks up npm env options?).

Edit: Ok, this doesn't quite work - npm calls neon build, which in turn calls cargo, which then calls npm; by this point we've dropped --devdir :( .

Edit2: Ensuring both are set seems to work (since cargo will pass on npm_config_devdir to npm). Still having issues with electron-rebuild refusing to actually trigger a native build...

[dherman]

Just a heads-up that we're making progress on this: you can see a simple demo at:

https://github.com/dherman/neon-electron-hello

It doesn't yet integrate with electron-rebuild but I've spoken with @paulcbetts, @MarshallOfSound, and @felixrieseberg, and we should be able to make that happen too. Until then, the demo uses a different command for building Neon dependencies than other native dependencies.

My next steps are to test on Windows and make sure that's working, and then also to make the tools smarter about rebuilding if someone has accidentally already built a dependency with the wrong ABI.

[intellild]

https://github.com/dherman/neon-electron-hello not working
I tried both electron-build-env and NEON_NODE_ABI=50
both my own project and the neon-electron-hello got the same error: (electron 1.4.16, ABI version is 50)

Module version mismatch.
Expected 50, got 48.

It seem that neon is not following the ABI version provided but using the ABI version of node(6.10.2, ABI version 48) that runs it?

[dherman]

Still would be good to integrate with electron-rebuild but for now I think we can call this working, and it's documented in the guides:

http://guides.neon-bindings.com/electron-apps/