FreeBSD Node support #678

hackacad · 2021-07-27T13:24:50Z

Signed-off-by: hackacad admin@hackacad.net

Description

Adding upstream support for FreeBSD

Issues Resolved

Using default path for node on FreeBSD

Signed-off-by: hackacad <admin@hackacad.net>

opensearch-ci-bot · 2021-07-27T13:26:24Z

✅ DCO Check Passed 745343c

dblock · 2021-07-27T16:26:25Z

Why wouldn't we be using the packaged version?

hackacad · 2021-07-27T16:37:22Z

The packaged version is Linux ELF and not compatible with FreeBSD

dblock · 2021-07-27T18:51:19Z

The packaged version is Linux ELF and not compatible with FreeBSD

Same question as for Java in OpenSearch. Would it be better if we made a FreeBSD package with the appropriate node version?

tmarkley · 2021-08-03T16:47:15Z

@hackacad thanks for the change! Can you respond to dB's question above?

Also, can we automate the testing for this?

hackacad · 2021-08-03T17:06:42Z

The FreeBSD port will always use the node shipped via FreeBSD (ports).
There're a couple of reasons (like one port for all release versions, not like on Linux Distros).

It would of course be possible to integrate one, but as you already mentioned this would require a full CI pipeline for testing on FreeBSD.

kavilla · 2021-08-31T04:31:10Z

Hello @hackacad, apologies on the delay. We will try to get some prioritization on this.

dblock · 2021-09-07T22:18:32Z

Seems pretty innocent to me. Bump @kavilla.

smortex · 2021-09-07T23:10:53Z

src/dev/build/tasks/bin/scripts/opensearch-dashboards

+
+if [ $OS = "freebsd" ]; then
+  NODE="/usr/local/bin/node"
+else
+  NODE="${DIR}/node/bin/node"
+fi
+


Another possibility (maybe simpler): if a bundled node is present, use it, otherwise expect one to bie found in $PATH:

Suggested change

if [ $OS = "freebsd" ]; then

NODE="/usr/local/bin/node"

else

NODE="${DIR}/node/bin/node"

fi

if [ -x "${DIR}/node/bin/node" ]; then

NODE="${DIR}/node/bin/node"

else

NODE="node"

fi

What about a
elif [ ! -x "$(which node)" ]; then
looks a little bit more solid then using node without a path. Not sure if has any side effects.

That's probably better and allow less modifications of the rest of the script.

@hackacad were you able to get insight getting node without the path.

For me, i think the original code for me I can pretty explicitly read that we are doing this conditional for FreeBSD node which makes me feel safer.

(basically the originally change is ok for me)

Using a more general condition looks better, but it should still look for a path. Using node without a path would cover most environments, but looking for it (like using which) should be much more solid.

kavilla · 2021-09-13T08:16:47Z

hackacad · 2021-09-17T20:00:32Z

src/dev/build/tasks/bin/scripts/opensearch-dashboards

+if [ $OS = "freebsd" ]; then
+  NODE="/usr/local/bin/node"
+else
+  NODE="${DIR}/node/bin/node"


NODE=which node

Just want to clarify here - are you planning on making additional changes?

Just added the improved and more liberal node selection (OS independent)

Signed-off-by: Sven R <admin@hackacad.net>

opensearch-ci-bot · 2021-09-21T16:30:08Z

✅ DCO Check Passed c80da20

src/dev/build/tasks/bin/scripts/opensearch-dashboards

Signed-off-by: Sven R <admin@hackacad.net>

opensearch-ci-bot · 2021-09-22T06:06:25Z

✅ DCO Check Passed eddeb2d

smortex · 2021-09-22T06:16:46Z

src/dev/build/tasks/bin/scripts/opensearch-dashboards

+elif [ -x "$(which node)" ]; then
+  NODE="$(which node)"
+fi
+
 test -x "$NODE"


Useless test. The same test happen 2 lines bellow in a condition. This line can be removed.

Signed-off-by: Sven R <admin@hackacad.net>

opensearch-ci-bot · 2021-09-22T06:34:00Z

✅ DCO Check Passed 23865dd

smortex

LGTM

tmarkley

This looks great! I appreciate the simplification.

Now that we have a more general solution, I think there are some final steps to close things out:

Change the title and description of the PR
Squash and amend the commits to reflect the changes that were made
Add details to the DEVELOPER_GUIDE to explain what is supported.

ananzh · 2021-09-29T16:21:53Z

@hackacad really appreciate the contribute. I am trying to launch a FreeBSD node in AWS but can't make it due to yarn & node version issue. Can you help us to test whether it will cause any backward compatibility issue to break the build? For example:

yarn build

then if you pull the newest commit, you should be able to run

yarn build-platform --linux

and

yarn build-platform --darwin

and etc.

Could you help us to test a bit more? I think other than this, we are good to go. Thanks.

kavilla · 2021-10-12T19:20:17Z

@hackacad any updates to what @ananzh asked?

Thanks!

hackacad · 2021-10-12T19:54:44Z

@hackacad any updates to what @ananzh asked?

Thanks!

Of course, we can support you in setting up an OpenSearch FreeBSD instance.
You can send an email to the FreeBSD team (opensearch@FreeBSD.org);
@stockholmux has access to our Discord channel as well

smortex · 2021-10-14T01:06:37Z

A quick message to add the link to the discord server where some members of the FreeBSD OpenSearch group are hanging out: OpenSearch Community (unofficial)

ananzh · 2021-10-15T06:39:26Z

@hackacad any updates to what @ananzh asked?
Thanks!

Of course, we can support you in setting up an OpenSearch FreeBSD instance. You can send an email to the FreeBSD team (opensearch@FreeBSD.org); @stockholmux has access to our Discord channel as well

Hello @hackacad thanks for the support. I have launched several freeBSD instances (freeBSD13-stable-amd and freeBSD11-stable-amd). To run yarn build or run dashboards, we need node version set to 10.24.1 and need to use yarn. In freeBSD13 and freeBSD11, the supported node versions are

pkg search node
kadnode-2.3.0                  P2P name resolution daemon
leafnode-1.11.12               NNTP package for offline news caching and reading
munin-node-2.0.67              Node-specific part of Munin
node-16.10.0                   V8 JavaScript for client and server
node-thrift-0.14.0             Node.js bindings for the Apache Thrift RPC system
node10-10.24.1_1               V8 JavaScript for client and server
node14-14.17.6                 V8 JavaScript for client and server
node_exporter-1.2.2            Prometheus exporter for machine metrics
npm-node14-6.14.8              Node package manager
p5-Tree-DAG_Node-1.32          Super class for representing nodes in a tree
p5-Tree-Node-0.08_2            Memory-efficient tree nodes in Perl
p5-WebService-Linode-0.29      Perl Interface to the Linode.com API
p5-XML-Node-0.11_1             Perl5 module to extend and simplify XML::Parser
p5-XML-NodeFilter-0.01_1       XML::NodeFilter is an object that know how to "filter out" nodes
py38-certbot-dns-linode-1.18.0 Linode DNS Authenticator plugin for Certbot
py38-node-semver-0.8.0         Python version of node-semver
py38-nodeenv-1.5.0             Node.js virtual environment builder
yarn-node14-1.22.11            Package manager for node, alternative to npm

The supported yarn versions are

pkg search yarn
yarn-1.22.11                   Package manager for node, alternative to npm
yarn-node14-1.22.11            Package manager for node, alternative to npm

The yarn-1.22.11 will remove node 10 and install node 16

Installed packages to be REMOVED:
node10: 10.24.1_1
 
New packages to be INSTALLED:
node: 16.10.0
yarn: 1.22.11

So in freeBSD, it seems dashboards can't be build directly. I would suggest use PR #795 and #840 as the references to add a build for freeBSD. For example, when run yarn build-platform --freeBSD a freeBSD artifact can be built in any system env. Then we could upload this freeBSD artifact from local env to a freeBSD instance then test it.

stockholmux · 2021-10-15T13:49:23Z

@ananzh can you use nvm to get node 10 for build purposes? (e.g. https://github.com/carlosgruiz-dev/carlosgruiz-dev/wiki/FreeBSD:-Install-nvm)

(should only be needed until we go v16)

ananzh · 2021-10-15T16:05:22Z

@ananzh can you use nvm to get node 10 for build purposes? (e.g. https://github.com/carlosgruiz-dev/carlosgruiz-dev/wiki/FreeBSD:-Install-nvm)

(should only be needed until we go v16)

Hey @stockholmux I tried this yesterday and it doesn't work. Here is the result:

root@freebsd:/home/ec2-user # pkg install gcc7 gmake
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
pkg: No packages available to install matching 'gcc7' have been found in the repositories

If search gcc7 by pkg search gcc7 there is no found. I think the best way is to build a freeBSD artifact in ubuntu with any related PRs then cp to freeBSD instance.

smortex · 2021-10-16T03:56:08Z

If search gcc7 by pkg search gcc7 there is no found. I think the best way is to build a freeBSD artifact in ubuntu with any related PRs then cp to freeBSD instance.

EOL node needs EOL gcc… legacy dependency hell 🙃

We (FreeBSD) avoid distributing software which has reached End Of Life. We currently support node 14 (LTS) and 16 (current), so I had to adjust this:

nvim ./package.json

diff --git a/package.json b/package.json
index b50a51e755..235d29a7d5 100644
--- a/package.json
+++ b/package.json
@@ -468,7 +468,7 @@
     "zlib": "^1.0.5"
   },
   "engines": {
-    "node": "10.24.1",
+    "node": "14.17.6",
     "yarn": "^1.21.1"
   }
 }

Then I could successfully run the commands you wrote:

sudo pkg install yarn-node14 python3 gmake
yarn osd bootstrap
yarn build-platform --linux
yarn build-platform --darwin

Also note that the FreeBSD port has this patch (not needed for building):
https://cgit.freebsd.org/ports/tree/textproc/opensearch-dashboards/files/patch-src_setup__node__env_node__version__validator.js

Not sure what you wanted to do @ananzh, hope this help! Running just yarn build I hit sass/sass#1873 (which I believe is not a FreeBSD problem).

Hey @smortex thanks for the confirm. I opened an issue here to remind us testing and verifying the build of dashboards once it is bumped to node 16. I will approve it. Thanks a lot for the contribution.

ananzh · 2021-10-18T22:05:18Z

@hackacad thanks again for the great work. would you like to merge the PR to main? I could help to backport it to 1.2 and 1.x.

kavilla · 2021-10-19T07:08:50Z

@ananzh,

probably could be backported just to the 1.x branch.

Backport PR: opensearch-project#678 Signed-off-by: hackacad <admin@hackacad.net>

Backport PR: #678 Signed-off-by: hackacad <admin@hackacad.net> Co-authored-by: hackacad <admin@hackacad.net>

FreeBSD Node support

745343c

Signed-off-by: hackacad <admin@hackacad.net>

hackacad mentioned this pull request Jul 27, 2021

Support FreeBSD opensearch-project/opensearch-build#101

Closed

smortex reviewed Sep 7, 2021

View reviewed changes

hackacad commented Sep 17, 2021

View reviewed changes

kavilla self-requested a review September 20, 2021 21:33

kavilla previously approved these changes Sep 20, 2021

View reviewed changes

more liberal node selection

c80da20

Signed-off-by: Sven R <admin@hackacad.net>

hackacad dismissed kavilla’s stale review via c80da20 September 21, 2021 16:27

tmarkley reviewed Sep 21, 2021

View reviewed changes

src/dev/build/tasks/bin/scripts/opensearch-dashboards Outdated Show resolved Hide resolved

consume less space

eddeb2d

Signed-off-by: Sven R <admin@hackacad.net>

smortex reviewed Sep 22, 2021

View reviewed changes

improved and simplified node test

23865dd

Signed-off-by: Sven R <admin@hackacad.net>

smortex approved these changes Sep 22, 2021

View reviewed changes

tmarkley reviewed Sep 22, 2021

View reviewed changes

ananzh self-requested a review September 29, 2021 16:35

ananzh approved these changes Oct 18, 2021

View reviewed changes

kavilla approved these changes Oct 19, 2021

View reviewed changes

kavilla merged commit 915a13c into opensearch-project:main Oct 19, 2021

ananzh pushed a commit to ananzh/OpenSearch-Dashboards that referenced this pull request Oct 20, 2021

[1.x] FreeBSD Node support

5001006

Backport PR: opensearch-project#678 Signed-off-by: hackacad <admin@hackacad.net>

ananzh mentioned this pull request Oct 20, 2021

[1.x] FreeBSD Node support #884

Merged

5 tasks

ananzh added a commit that referenced this pull request Oct 21, 2021

[1.x] FreeBSD Node support (#884)

18e1784

Backport PR: #678 Signed-off-by: hackacad <admin@hackacad.net> Co-authored-by: hackacad <admin@hackacad.net>

ananzh added the v1.2.0 label Nov 1, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FreeBSD Node support #678

FreeBSD Node support #678

hackacad commented Jul 27, 2021

opensearch-ci-bot commented Jul 27, 2021

dblock commented Jul 27, 2021

hackacad commented Jul 27, 2021

dblock commented Jul 27, 2021

tmarkley commented Aug 3, 2021

hackacad commented Aug 3, 2021

kavilla commented Aug 31, 2021

dblock commented Sep 7, 2021

smortex Sep 7, 2021

hackacad Sep 13, 2021

smortex Sep 13, 2021

kavilla Sep 17, 2021 •

edited

Loading

hackacad Sep 17, 2021

kavilla commented Sep 13, 2021

hackacad Sep 17, 2021

tmarkley Sep 21, 2021

hackacad Sep 21, 2021

opensearch-ci-bot commented Sep 21, 2021

opensearch-ci-bot commented Sep 22, 2021

smortex Sep 22, 2021 •

edited

Loading

opensearch-ci-bot commented Sep 22, 2021

smortex left a comment

tmarkley left a comment

ananzh commented Sep 29, 2021

kavilla commented Oct 12, 2021

hackacad commented Oct 12, 2021

smortex commented Oct 14, 2021 •

edited

Loading

ananzh commented Oct 15, 2021

stockholmux commented Oct 15, 2021 •

edited

Loading

ananzh commented Oct 15, 2021 •

edited

Loading

smortex commented Oct 16, 2021 •

edited by ananzh

Loading

ananzh commented Oct 18, 2021

kavilla commented Oct 19, 2021

FreeBSD Node support #678

FreeBSD Node support #678

Conversation

hackacad commented Jul 27, 2021

Description

Issues Resolved

opensearch-ci-bot commented Jul 27, 2021

dblock commented Jul 27, 2021

hackacad commented Jul 27, 2021

dblock commented Jul 27, 2021

tmarkley commented Aug 3, 2021

hackacad commented Aug 3, 2021

kavilla commented Aug 31, 2021

dblock commented Sep 7, 2021

smortex Sep 7, 2021

Choose a reason for hiding this comment

hackacad Sep 13, 2021

Choose a reason for hiding this comment

smortex Sep 13, 2021

Choose a reason for hiding this comment

kavilla Sep 17, 2021 • edited Loading

Choose a reason for hiding this comment

hackacad Sep 17, 2021

Choose a reason for hiding this comment

kavilla commented Sep 13, 2021

hackacad Sep 17, 2021

Choose a reason for hiding this comment

tmarkley Sep 21, 2021

Choose a reason for hiding this comment

hackacad Sep 21, 2021

Choose a reason for hiding this comment

opensearch-ci-bot commented Sep 21, 2021

opensearch-ci-bot commented Sep 22, 2021

smortex Sep 22, 2021 • edited Loading

Choose a reason for hiding this comment

opensearch-ci-bot commented Sep 22, 2021

smortex left a comment

Choose a reason for hiding this comment

tmarkley left a comment

Choose a reason for hiding this comment

ananzh commented Sep 29, 2021

kavilla commented Oct 12, 2021

hackacad commented Oct 12, 2021

smortex commented Oct 14, 2021 • edited Loading

ananzh commented Oct 15, 2021

stockholmux commented Oct 15, 2021 • edited Loading

ananzh commented Oct 15, 2021 • edited Loading

smortex commented Oct 16, 2021 • edited by ananzh Loading

ananzh commented Oct 18, 2021

kavilla commented Oct 19, 2021

kavilla Sep 17, 2021 •

edited

Loading

smortex Sep 22, 2021 •

edited

Loading

smortex commented Oct 14, 2021 •

edited

Loading

stockholmux commented Oct 15, 2021 •

edited

Loading

ananzh commented Oct 15, 2021 •

edited

Loading

smortex commented Oct 16, 2021 •

edited by ananzh

Loading