No control over the key used for the generated importmap

[simontaurus]

Description

Import map key generation seems to be based on the dependencies filename:

nicegui/nicegui/dependencies.py

Line 126 in 09f4c34

return path.name.split('.', 1)[0]

e.g.

dependencies=['node_modules/@json-editor/json-editor/dist/jsoneditor.js']

leads to the key jsoneditor which is also required in dynamic imports within custom vue components:

import("jsoneditor")

(see also #4162)

However, this may lead to conflicts if two libs use the same filename. Also, it would be preferable to use the correct package name to stay interoperable with other environments (e.g. vite):

import("@json-editor/json-editor")

[falkoschindler]

Thanks for reporting this issue, @simontaurus!
And sorry for keeping you waiting. Now I finally found the time and focus to look into it.

You're absolutely right: Basically using the file name for the importmap key is very fragile and can lead to conflicts rather quickly. I'm just not sure how to improve it.

This is what _get_name is called with when running the NiceGUI website:

/Users/falko/Projects/nicegui/nicegui/elements/aggrid.js
/Users/falko/Projects/nicegui/nicegui/elements/lib/aggrid/ag-grid-community.min.js
/Users/falko/Projects/nicegui/nicegui/elements/audio.js
/Users/falko/Projects/nicegui/nicegui/elements/mermaid.js
/Users/falko/Projects/nicegui/nicegui/elements/lib/mermaid/mermaid.js
/Users/falko/Projects/nicegui/nicegui/elements/lib/mermaid/mermaid.min.js
/Users/falko/Projects/nicegui/nicegui/elements/lib/mermaid/mermaid.esm.min.mjs
/Users/falko/Projects/nicegui/nicegui/elements/lib/mermaid/chunks/mermaid.esm.min/blockDiagram-NDWNTGEE.mjs
/Users/falko/Projects/nicegui/nicegui/elements/lib/mermaid/chunks/mermaid.esm.min/c4Diagram-AUYESYAG.mjs
/Users/falko/Projects/nicegui/nicegui/elements/lib/mermaid/chunks/mermaid.esm.min/chunk-2RSIMOBZ.mjs
...
/Users/falko/Projects/nicegui/nicegui/elements/lib/mermaid/chunks/mermaid.esm.min/xychartDiagram-RLS75X5Z.mjs
/Users/falko/Projects/nicegui/nicegui/elements/markdown.js
/Users/falko/Projects/nicegui/nicegui/elements/timer.js
/Users/falko/Projects/nicegui/nicegui/elements/codemirror.js
/Users/falko/Projects/nicegui/nicegui/elements/colors.js
/Users/falko/Projects/nicegui/nicegui/elements/dark_mode.js
/Users/falko/Projects/nicegui/nicegui/elements/echart.js
/Users/falko/Projects/nicegui/nicegui/elements/lib/echarts/echarts.min.js
/Users/falko/Projects/nicegui/nicegui/elements/lib/echarts-gl/echarts-gl.min.js
/Users/falko/Projects/nicegui/nicegui/elements/editor.js
/Users/falko/Projects/nicegui/nicegui/elements/fullscreen.js
/Users/falko/Projects/nicegui-highcharts/nicegui_highcharts/highchart.js
/Users/falko/Projects/nicegui-highcharts/nicegui_highcharts/lib/highcharts/highcharts.js
/Users/falko/Projects/nicegui-highcharts/nicegui_highcharts/lib/highcharts/highcharts-3d.js
/Users/falko/Projects/nicegui-highcharts/nicegui_highcharts/lib/highcharts/highcharts-more.js
/Users/falko/Projects/nicegui-highcharts/nicegui_highcharts/lib/highcharts/modules/accessibility.js
...
/Users/falko/Projects/nicegui-highcharts/nicegui_highcharts/lib/highcharts/modules/xrange.js
/Users/falko/Projects/nicegui/nicegui/elements/image.js
/Users/falko/Projects/nicegui/nicegui/elements/input.js
/Users/falko/Projects/nicegui/nicegui/elements/interactive_image.js
/Users/falko/Projects/nicegui/nicegui/elements/joystick.vue
/Users/falko/Projects/nicegui/nicegui/elements/joystick.vue
/Users/falko/Projects/nicegui/nicegui/elements/lib/nipplejs/nipplejs.js
/Users/falko/Projects/nicegui/nicegui/elements/json_editor.js
/Users/falko/Projects/nicegui/nicegui/elements/lib/vanilla-jsoneditor/standalone.js
/Users/falko/Projects/nicegui/nicegui/elements/keyboard.js
/Users/falko/Projects/nicegui/nicegui/elements/leaflet.js
/Users/falko/Projects/nicegui/nicegui/elements/link.js
/Users/falko/Projects/nicegui/nicegui/elements/notification.js
/Users/falko/Projects/nicegui/nicegui/elements/plotly.vue
/Users/falko/Projects/nicegui/nicegui/elements/plotly.vue
/Users/falko/Projects/nicegui/nicegui/elements/lib/plotly/plotly.min.js
/Users/falko/Projects/nicegui/nicegui/elements/query.js
/Users/falko/Projects/nicegui/nicegui/elements/scene.js
/Users/falko/Projects/nicegui/nicegui/elements/lib/three/three.module.js
/Users/falko/Projects/nicegui/nicegui/elements/lib/three/modules/BufferGeometryUtils.js
...
/Users/falko/Projects/nicegui/nicegui/elements/lib/three/modules/STLLoader.js
/Users/falko/Projects/nicegui/nicegui/elements/lib/tween/tween.umd.js
/Users/falko/Projects/nicegui/nicegui/elements/scene_view.js
/Users/falko/Projects/nicegui/nicegui/elements/lib/tween/tween.umd.js
/Users/falko/Projects/nicegui/nicegui/elements/lib/three/three.module.js
/Users/falko/Projects/nicegui/nicegui/elements/select.js
/Users/falko/Projects/nicegui/nicegui/elements/table.js
/Users/falko/Projects/nicegui/nicegui/elements/teleport.js
/Users/falko/Projects/nicegui/nicegui/elements/input.js
/Users/falko/Projects/nicegui/nicegui/elements/upload.js
/Users/falko/Projects/nicegui/nicegui/elements/video.js
/Users/falko/Projects/nicegui/nicegui/functions/refreshable.js
/Users/falko/Projects/nicegui/website/documentation/intersection_observer.js

I don't see a generic way to find the library name like "three" or "highcharts" or "nicegui". Or should we simply hash the file path? But how would components like joystick.vue know how to import their dependencies if the prefix isn't deterministic?

await import("???/nipplejs");

[evnchn]

Couple of notable points:

1. We were thinking to straight-up fetch the files from node_modules while working on Pin version for JS dependencies #4632. Perhaps we do one step further and eliminate the middleman and have NiceGUI clients access the node_modules folder?
2. If not, npm.py can put like a .libraryname file in the directory it just dumped files to, so that compute_key can take that as inspiration
3. Should point 1 be achieved, it re-ignites the possibility of Server-Side Rendering I was mentioning in Server-Side Rendering by invoking NodeJS, Vue and Quasar for the HTML #4557

In conclusion: to resolve this issue needs hand-in-hand with modification to how JS dependencies are managed. Straight-up patching dependencies.py likely lands us nowhere.

[falkoschindler]

We were thinking to straight-up fetch the files from node_modules

Interesting thought. Then we could use a .gitignore to filter the files we don't need in the repo and PyPI package.

But how would the user be able to add custom components with their own dependencies? Ideally we have one process that works for both, built-in libraries and custom ones.

[evnchn]

But how would the user be able to add custom components with their own dependencies?

I'd imagine the user can "mount" their own node_modules folder. Files are served from the user's node_modules if found, if not then NiceGUI's, if not then 404.

Benefit being easy addition of new components, and can override the NiceGUI's library versions, in case an urgent need to upgrade to a newer version of JS dependency arises.

[falkoschindler]

I'm just collecting issues and PRs for the 3.0 milestone (currently scheduled for end of August).
This issue seems like it might need breaking changes in the dependency management. Or we keep it simple, find a non-breaking solution and address custom components later. What do you think?

[simontaurus]

What about

class Dependency:
  name: str
  path: str

dependencies: List[Union[str, Dependency]]

dependencies=['node_modules/@json-editor/json-editor/dist/jsoneditor.js']  # legacy
dependencies=[Dependency(name='@json-editor/json-editor', path='node_modules/@json-editor/json-editor/dist/jsoneditor.js')]  # explicit name, legacy file path handling
dependencies=[Dependency(name='@json-editor/json-editor')]  # auto-install and reference from `node_modules`

[falkoschindler]

Interesting idea, @simontaurus!

Currently dependencies is of type List[Union[str, Path]], so we'd need to include Path for full compatibility.

But importing and using the Dependency class feels a bit cumbersome. Maybe we can simply add support for dictionaries?

dependencies = ['node_modules/@json-editor/json-editor/dist/jsoneditor.js']  # legacy
dependencies = {'json-editor': 'node_modules/@json-editor/json-editor/dist/jsoneditor.js'}  # explicit name, legacy file path handling
dependencies = {'json-editor': Path('node_modules/@json-editor/json-editor/dist/jsoneditor.js')}  # explicit name, legacy file path handling
dependencies = {'json-editor': ...}  # auto-install and reference from `node_modules`

To be honest, using ... to indicate auto-install dependencies feels a bit sneaky...

Anyway, such a backward-compatible solution would take pressure off the 3.0 release.

[falkoschindler]

This issue has been resolved in PR #5021 which will be release in version 3.0.