[feat] Napi support package c++ static binary.

[shibd]

Motivation

Currently, users using the node js client must first install the C++ client. And when the user executes npm install, it also needs to be compiled. This is not user-friendly.

This PR is to provide pre-built packages for all the supported environments, using GYP and rebuild. The pre-built package is then uploaded to Github Release, and when the user installs the client, the system version is matched and the pre-built package is downloaded.

Currently tested linux_glibc_x86_64, linuc_musl_x86_64, linux_glibc_arm64, linuc_musl_arm64 and mac_x86_64. I've built it in my fork repository and can install it directly using the following command (without first installing the C++ client).

npm install shibaodi-pulsar-client

Modifications

New Github workflows

• When trigger release:
  1. Build the binary C++ static library first(use new c++ client 3.0 source).
  2. And use node-pre-gyp pre-built napi packages.
  3. Use node-pre-gyp-github publish this package to GitHub Release.
• When trigger unit test(use ubuntu GitHub runner):
  1. Install c++ lib(use new c++ client 3.0 version).
  2. Use this c++ lib build napi.
  3. Use pulsar image start broker(use pulsar latest).
  4. Run the unit test.

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

TODO

• Add pre-build windows binary package.
• Change user docs.

[BewareMyPower]

Why do you rename the directory name from certificate to test-conf?

[shibd]

Why do you rename the directory name from certificate to test-conf?

Just for the unified, put all the test-related configurations together.

[BewareMyPower]

Just for the unified

There is no need to make two separate repositories consistent. What if the Python client repo renamed the directory or adjust the hierarchy later? In this case, do you also want to sync the changes here? People who are interested in Node.js client might not be interested in Python client. That's why do we separate C++/Python repos.

From my perspective, if you have a huge PR, please remove unnecessary changes as much as possible.

[shibd]

From my perspective, if you have a huge PR, please remove unnecessary changes as much as possible.

Make sense. I removed unnecessary changes. PTAL.

[shibd]

Currently completed:

• linux_glibc_x86_64
• linux_musl_x86_64
• linux_glibc_arm64
• linux_musl_arm64
• macOS_x86_64

Please refer to my repo release: https://github.com/shibd/pulsar-client-node/releases/tag/v1.8.1-rc.3

Other environments:

• macOS_arm64: Not supported at the moment. Although the cpp client can be properly compiled on the x86_64 by cross-compilation, it is not successful when packaging NAPI. Maybe we need a macOS_m1 runner to run the compilation.
• windows_x86_64: I'm doing compilation testing, which will be added later.

@merlimat Do you feel there are other environments that need support?

@BewareMyPower @merlimat This PR can be reviewed first when you have time. Thanks.

[merlimat]

Very nice to see such progress on the packaging!

[merlimat]

macOS_arm64: Not supported at the moment. Although the cpp client can be properly compiled on the x86_64 by cross-compilation, it is not successful when packaging NAPI. Maybe we need a macOS_m1 runner to run the compilation.

We can try to find a way to cross-compile. Right now there are no M1 runners in GH actions.
Let this not be a blocker.

[shibd]

We can try to find a way to cross-compile. Right now there are no M1 runners in GH actions.
Let this not be a blocker.

Hi, @merlimat I can cross-compile libpulsarwithdeps.a, but I can't cross-compile napi. I tried using the command --target_arch=arm64, but it doesn't work. You can refer to failed job: https://github.com/shibd/pulsar-client-node/actions/runs/3294556450/jobs/5432190252

I raised an issue here and haven't waited for a response yet.

I'll finish compiling Windows first and then continue working on it here.

If you have a good idea, please recommend it to me. Thanks.

[merlimat]

👍 Looks good! Let's target windows build in a new PR