LaserWeb4 development, Linux, the universe and everything

[janschiefer]

LaserWeb4 seems to have dropped support for Linux a while ago. This is a major problem for many users, since LaserWeb is the only "serious" laser engraving solution for GRBL users out there.

However I cannot really comprehend this, since I am able to run the program without major issues on Linux, using the current development branch dev-es6 from 12th April of 2020.

Just run:
npm run installdev (installs all dependcies)
npm start (compiles and starts the server + app)

...and you are ready to go.

The only thing I have to figure out yet, is how to build an proper AppImage or .deb from the current development branch...

Any comments or suggestions?

BTW: I'm using Ubuntu Focal beta pre-release.

[cprezzi]

You are very welcome to contribute and care about the AppImage! Unfortunately most devs have left the project and I don't know much about linux.

[janschiefer]

Ok, why not.

Unfortunately I don´t have much experience with the AppImage ecosystem (and also I am a physician in COVID-19 stress). However, Linux and CNC is my hobby for relaxation.

Anyone know a good "quick and dirty" tutorial for packaging nodejs-Applications into .AppImage or .deb?

[cprezzi]

A good starting point should be
https://docs.appimage.org/packaging-guide/index.html

[cprezzi]

You should also check https://github.com/LaserWeb/lw.comm-server, especially the scripts in package.json and the docker files.

[mathematicalmichael]

Might I suggest packaging it into a docker image? Would probably be much easier than packaging it as a deb.

[janschiefer]

Trying to build the AppImage.
I'm having the LaserWeb4 and lw.comm-server in parallel directories.

(...)
packaging for linux x64 using electron 2.0.18 to dist/linux-unpacked
Unhandled rejection Error: Exit code: 1. Command failed: gm convert -size 48x48 /tmp/electron-builder-N3DabI/0-2-linux.iconset/icon_128x128x32.png -resize 48x48 /tmp/electron-builder-N3DabI/0-2-linux.iconset/icon_48x48x32.png
gm convert: Unable to open file (/tmp/electron-builder-N3DabI/0-2-linux.iconset/icon_128x128x32.png) [File not found].

gm convert: Unable to open file (/tmp/electron-builder-N3DabI/0-2-linux.iconset/icon_128x128x32.png) [File not found].
(...)

Am I missing a git submodule or repository with the image/icon resource-files...?

[janschiefer]

Ok.
I have created working builds of the latest git version of LaserWeb4 for Linux x64 and Windows x64 with a little hacking: https://drive.google.com/open?id=1oGcQKB0K32n3B7irCJjCIg7AIyqRk-hz

• I removed all references to any icon files from package.json in lw.comm-server, so the default electron-builder icon is used.

• I created a symlink to "libreadline.so.6" (this version is not included in Ubuntu 20.04 Beta) from my system libreadline.so.

You have to manually open 127.0.0.1:8000 in your browser to open the GUI in this build.

[janschiefer]

I'll try to create a patch for the build problems in the next few days.

How can I submit the patch?

[deanforbes]

will this be able to run on a raspberry pie ? would it be possible to test it on that ?

[cprezzi]

There is a installation guide for rasperry pi on the wiki!
https://github.com/LaserWeb/LaserWeb4/wiki/1.4-Installation-on-RaspberryPi

The server part (lw.comm-server) can run on a raspberry pi without problems, but the web frontend needs a lot of memory and should be run on a real pc (linux, win, osx).

[danielkucera]

What about using Docker? #595

[zettrik]

Thank you Daniel, it's ist working fine with Docker - also on raspberry pi.

[aquilarubra]

Hi, I am on Arch and the docker way did not work, in that it cannot read the devices. I tried to run with --privileged, specifying the dev, and some other ways, but none worked. I suspect it does not work on Linux at all.

So, I investigated other ways and came up with a few solutions. At the end, I also built some Arch PKGBUILDs (for latest tag and git) that automatically builds the appimage and integrate it into Arch environment.

I noticed that something was taken from the janschiefer's pull request, but all the icons are still missing. Maybe that pull should be rechecked.

In my appimage version, browser opens automatically together with a console for the server. There are situations in which one closes the browsers and the server still runs in the background, with no way to close it. So, I found it better to have it visible. I preferred to integrate the appimage directly in Linux system rather than rely on its integration method.

However, one can also launch the appimage alone, and despite not being integrated it will launch the browser. I add a small modification to server.js to achieve that.

So, here are my notes, which can be applied to other Linux systems also (maybe something can be integrated in the documentation):

LaserWeb4 on Arch

1. Install nvm
   yay -S nvm

2. Init nvm
   source /usr/share/nvm/init-nvm.sh

3. Install nodejs >= 10.15.1 and < 11.x
   nvm install 11

4. Use nodejs v11
   nvm use 11

5. Clone the LaserWeb4 repo
   cd /usr/src
   git clone https://github.com/LaserWeb/LaserWeb4.git
   cd LaserWeb4

6. Install all dependencies
   npm run installdev (installs all dependcies)

7. Compile and start the server + app
   npm start

RUN: To run the server + app only
source /usr/share/nvm/init-nvm.sh
nvm use 11
npm run start-server

Appimage creation via electron-builder

(https://github.com/LaserWeb/lw.comm-server/pull/80/files)

1. Install electron
   pacman -S electron xorriso

2. Clone near LaserWeb4
   cd /usr/src
   git clone https://github.com/LaserWeb/LaserWeb4.git # If not already done
   git clone https://github.com/LaserWeb/lw.comm-server.git
   cd lw.comm-server

3. Copy missing app-icons in app

4. Install electron-builder
   source /usr/share/nvm/init-nvm.sh
   nvm use 11
   npm install ncp
   npm install electron
   npm install electron-builder

5. Add automatic browser opening after server launch
   echo -e "require('dns').lookup(require('os').hostname(), function (err, add, fam) {\n var start = (process.platform == 'darwin'? 'open': process.platform == 'win32'? 'start': 'xdg-open');\n require('child_process').exec(start + ' http://' + add + ':' + config.webPort);\n});\n" >> server.js

6. Fix libraries
   ln -s /usr/lib/libreadline.so.8.0 /usr/lib/libreadline.so.6

7. Build appimage (saved in dist)
   npm run dist

RUN: To run the server + app only
source /usr/share/nvm/init-nvm.sh
nvm use 11
node server.js

Tools

• To kill the laserweb server that was not started from a console
  kill $(netstat -ltpn | grep -w ':8000' | awk '{print $7}' | cut -d/ -f1)

• To uninstall the laserweb appimage
  find /home/$(whoami)/.local/share -type f -name 'appimagekit-lw.comm-server*' -delete

PKGBUILDs for Arch

laserweb4.tar.gz
laserweb4-git.tar.gz

[iamajoe]

I've just created a pull request to fix the dockerfile #647
After merge, we could setup it on dockerhub for example and easily run it with volume linkage (for data persistence) even on a homelab setup. If you just care about the gcode generation, you could also just run it on a small server instance on linode / digitalocean...

The package server is indeed flawed. It seems node-gyp isn't able to have access to the system ports and fails, even with root permissions.

[cprezzi]

@joesantosio I've merged your pr #647
Could you set it up on dockerhub? I have no clue about docker ;)

About the system ports. Could this be caused by missing/incompatible libusb and libudev, as @QuirkyCort wrote in issue #648?

[iamajoe]

@cprezzi so I've created the docker image under my user. We can change it for a laserweb account but for that you will need to create the laserweb dockerhub account, create the access token and add it to the secrets here on the repository as DOCKERHUB_TOKEN and the username as DOCKERHUB_USERNAME. For now, I am running the CI on my fork on the branch prod which is intended to be the release branch.

I set up a PR to modify the README.md as per the changes ( #650 ).

Did anyone test a higher version of node? 10 is quite low and it should be bumped.

[harlock999]

Right now i'm running on @easytarget fork that has newer nodejs version with good success. My understanding is it was quite some work. Idea is eventually pull easytarget version back into the main LaserWeb4.

@easytarget : Any idea when you'll release you'll issue a pull request?

[cprezzi]

Sure, when the @easytarget fork is ready we will pull it over to LW.

[iamajoe]

@cprezzi by the way, I don't have other way to communicate with you, I am starting the work on a decoupled API in Go.

[cprezzi]

@cprezzi so I've created the docker image under my user. We can change it for a laserweb account but for that you will need to create the laserweb dockerhub account, create the access token and add it to the secrets here on the repository as DOCKERHUB_TOKEN and the username as DOCKERHUB_USERNAME. For now, I am running the CI on my fork on the branch prod which is intended to be the release branch.

I set up a PR to modify the README.md as per the changes ( #650 ).

Did anyone test a higher version of node? 10 is quite low and it should be bumped.

@joesantosio The actual servrer uses node 12 and it also seems to be working with node 18 (according to #648).

[cprezzi]

@joesantosio Why do you think we need a go version?
The node js version is very versatile and I can't support go.

[iamajoe]

First of all, I don't think we need per se a Go version and take all of the next rant as an opinion and nothing more than that.

In my opinion, we do need a fully decoupled backend from the frontend. The communication is already in place for this to happen and it makes it easier and more flexible to have that separation. You can have the frontend wherever you like running wherever you like and the backend, could run even on a smaller microcontroller (theoretically). Being decoupled also means that the server doesn't depend so much on the frontend and vice-versa which means that you can make better changes on either stack, you can assign people for more frontend or more backend and things tend to evolve a bit faster in that environment because the modularization helps to understand the development thinking.

Since, in my opinion, this is a good change, better to do it "right" this time around (not saying it is "bad", just that it could be "better".

Architecture is one thing where I think things could be improved. A pattern similar to DDD for example for better separation of concerns and modularization in general.
The language is other topic where I think it could be improved. JavaScript by nature is flawed. Its dynamic abilities is also one of the cancers of the language. It opens to insecurity in the code specially while developing bugs arise more often and you are never really in a position that you can confidently state that your code works. The "node_modules" folder hell and dependencies in general is other cancer. There are a million of horror stories and I bet you have yours.
With that in mind, in my opinion, there are 2 contenders for this spot. TypeScript and Go. I work everyday with the 2 by the way.
Both are compiled (even though TypeScript can also be a transpilation layer to JavaScript), both have typing and in general both are better development experience.
TypeScript, as kind of a superset of JavaScript, also inherits its flaws. The dependency hell, you can circumvent the safety, if used as a transpilation layer its dynamic nature that might not work somewhere else or even if compiled might be a bit of a challenge somehow with some dependencies, node in general is harder to deploy.
Go is strongly typed, compiles to a single binary file that doesn't need anything else to run, the language itself is quite easy to understand and learn in 1 week, no magic involved, it is a more robust language, it doesn't need 2gb of memory to run, 100mb is just fine or even less, I prefer Go C binds, a testing framework included, a good standard library with probably most of what you need, the Go community doesn't do dependency packages just because (there are no 3 line packages like in the node world and for some reason are used everywhere?!) and quite a bunch of smart people, easy compilation, good tooling...

I just find Go to be a better overall language and experience for the backend side of things.
Sorry about the long explanation. Tried my best.

[iamajoe]

This all said, I am just dwelling with that idea of Go. I do think, again in my opinion, that at least TypeScript should be used but probably the first steps would be to setup the architecture, decouple, modularize and unit tests so it stays stable regardless of the language.

[cprezzi]

The backend is alredy decouppled. It just has the option to embed the web frontend and serve it via http. Only socket.io is a dependency that both (frontend and backend) need, everything else is independant.

If you want a pure backend, you can just drop the http server and delete the app folder.

But I prefer to have the option of an embedded website for LaserWeb or any other frontend, because it is very convenient for the user to not have to install any frontend app and just use a webbrowser. It also gives us the option to compile a standalone exe (frontend with integrated backend) with electron.

I would also like to generate a version with an embedded generic "machine control" or "gcode sender" frontend (like octo-pi).

The backend is quite simple. Basically a socket to serial gateway. I don't see the need for changing much on the backend side. Ok, it's not modular at all, but it's quick to maintain (at least for me ;) ).

I see much more need for maintenance and development in the frontend. @easytarget already started a new version with updated dependancies and additional features. We need to discuss what's next when his version is releasd.

[iamajoe]

Yes I can see the advantage of having the same package with all. I think there are ways though to still maintain the all-in-one package with both decoupled.

Well... it isn't really that easy to maintain as is :p but I understand you. I was thinking on just making a simple PR proposition on how I see this working (foregoing the Go and language change at the first instance) but since someone else is already working on a reviewed version, starting from there makes perfect sense.