NPM / Libraries

[aersam]

Hi Guys

NiceGui handles dependencies on external scripts in a very simple way which is not only bad, but has a number of drawbacks. NiceGui currently ships the js files of external dependencies as part of the pypi package. Meaning each included component will increase the package size more and more. In addition, fixes of included JavaScript Libraries will need to be pushed by an update of NiceGui which is not so great, especially for security fixes. So I think one should think about better ways even if they introduce complexity.
Also bundling and minification are lacking which would optimize production load time

So, I thought I just write down a little proposal to start a discussion about it :)

Preconditions:

• No need for NodeJS
• Bundling / Minification should be optional / Step 2
• Rely on ESM Modules only. We're in 2023 ;)

Proposal

1. Instead of including the JS Files in the pypi Package, add some FastAPI Route which acts as a proxy to jsdelivr or npm directly. The first time a package is requested, it's downloaded to some cache folder and the js file(s) are served. There should be a CLI or so to download the Packages during build instead. There needs to be an allow-list for the packages that can be requested by frontend, and I would only allow one specific version for each library.

In a python component, one would declare the dependency something like this:

class AgGrid(Element, component='aggrid.js', libraries=['npm:ag-grid-community@30']):

In JavaScript:

import { Grid } from 'ag-grid-community';

To map 'ag-grid-community' to the correct esm js file without a bundler, one can use metadata from package.json ("module" field) and import maps

2. For advanced uses / performance, allow bundling using esbuild. Esbuild could be invoked using subprocess and is very, very fast and does not require NodeJS

[falkoschindler]

Very interesting idea, @aersam! We should definitely improve the dependency management (again). And the growing number of third-party libraries concerns me as well.

We already thought a similar direction and ended up with concerns about the following scenario: Imagine you develop a robot software and want to deploy to a new machine. You install NiceGUI, maybe in a Docker container, and maybe even start the app and check if it is running. The dependencies are loaded and cached. Then you move the robot into the field where there might be no internet access. Now someone wants to access some performance plot on a separate analysis page, but Plotly isn't available because it has never been cached.

So we might need to give the user control over when to load which dependencies. Meanwhile NiceGUI shouldn't get more complicated for unexperienced users that like its simplicity. But I'm sure, we can find an API that is both, nice and powerful.

[aersam]

The offline case is very valid and I have the same thing on my azure website because some friendly security guy blocks almost all traffic. I will have to download the dependencies during build. I think there is need for documentation about this, but a simple CLI should do it. I think we should have an npm caching directory which you can set by some environment variable. You can set the environment variable during CI/CD build to download packages there and set it to the same directory during production.

Offline is however an advanced case I guess and somethink that's ok to be a bit more involved - as long as it's documented clearly. Also bundling/minification is a bit advanced, but would solve it, too