Include @jupyterlab/builder in this package

[fcollonval]

We should bring @jupyterlab/builder within this package and make it generic.

For now the code lives there: https://github.com/jupyterlab/jupyterlab/tree/main/builder

[jtpio]

After the move, should the package still be named @jupyterlab/builder, or @jupyter/builder?

Since the Python package seems to be named jupyter_builder at the moment:

https://github.com/jupyterlab/jupyterlab-builder/blob/c04e4a2523df1ccbfc2d4bf9edb34cdd2bd25fb3/pyproject.toml#L9

Looks like @jupyterlab/builder may be preferable to avoid changing extensions that depend on it. Although there might be some ways to handle renames of npm packages without too much disruption.

[fcollonval]

@cronan03 to test this, you need to

1. Bring the Typescript files and the package.json
2. Build the Javascript assets; normally it should create a lib folder when you execute jlpm run build
3. You should check that the code is calling those Javascript assets in that Python function:

https://github.com/jupyterlab/jupyterlab-builder/blob/c04e4a2523df1ccbfc2d4bf9edb34cdd2bd25fb3/jupyter_builder/federated_extensions.py#L215

That implies defining properly the core_path to get the correct builder variable (it should point to the lib/build-labextension.js file)

[Darshan808]

@fcollonval
I’m interested in working on this and have already made some progress. I’d like to share what I’ve done so far and also have a few questions. I sent an email but haven’t received a response yet.

[Darshan808]

Hey @fcollonval,

I’ve gone through the issue as thoroughly as I could and have a basic understanding of it. However, I still have a few questions that I believe are crucial for me to move forward. I transitioned builder from @jupyterlab/builder to jupyter-builder.

Could you kindly clarify the following:

1. Why are we checking the compatibility of versions of @jupyterlab/builder of core and as of dependency? The required builder version is already specified in the extension’s package.json, so it will be downloaded as a dependency. How does the core’s builder version play into this?

2. Am I approaching this correctly by running build-labextension.js from the core jupyter-builder? I now think that we need to add jupyter-builder/builder as a dependency in the extension, install it in node_modules, and then run node_modules/jupyter-builder/builder/lib/build-labextension.js to build the extension. In this case, the --core-path would point to the actual core jupyter-builder folder, where we check for version compatibility between the core and the dependency (though I still don’t fully understand why this is necessary).

I’d really appreciate your insights on this. It will help me overcome these roadblocks and push the work forward.

[Darshan808]

@fcollonval
Sketched a simple chart of my understanding!
Am I on right track ?

[fcollonval]

Hey @Darshan808
Thanks for pushing on this.

The @jupyterlab/builder code contains the webpack configuration; see https://github.com/jupyterlab/jupyterlab/blob/main/builder/src/webpack.config.base.ts
This is the reason we are checking the core and an extension have matching version to ensure webpack config compatibility.

Regarding the interaction between the python code and the js code. As you found, it is python that executes the js code. You can see that more clearly looking at the extension building script:
https://github.com/jupyterlab/extension-template/blob/5696924f5b7adf07cf07ea9d3510c66914fbb770/template/package.json.jinja#L35

So about what will happen for extensions is the following:
The build dependency at
https://github.com/jupyterlab/extension-template/blob/5696924f5b7adf07cf07ea9d3510c66914fbb770/template/pyproject.toml.jinja#L2
should change to replace juputerlab by jupyter-builder.
Explicitly defining, @jupyterlab/builder in the package.json should not be needed.
But then the challenge is to get the core configuration, in particular we need to know which packages are provided by the core and should not be bundled within the extension. That information lives there
https://github.com/jupyterlab/jupyterlab/blob/84572d6cea7eb73f2cfb2a0cc0cec4387b3462be/dev_mode/package.json#L285
Unfortunately it is not published.

So I would say, the first step is to get the JavaScript code in. And ensure the python code is using the build-labextension file from jupyter-builder and not Jupyterlab.

Then we will test how to modify extension build settings to use jupyter-builder.
Finally, we will address the issue of getting the core packages list without requiring to install JupyterLab.

[fcollonval]

Do those comments make sense?

[Darshan808]

@fcollonval

Progress Update

Here is the progress I've made so far:

1. Moved the builder folder to jupyter-builder.
2. Updated the package.json in the extension template to include @jupyter-builder/builder as a development dependency.
3. Modified the template's scripts to use jupyter-builder on running jlpm run build, which calls build-labextension.js from the @jupyter-builder/builder package in the extension's node modules.

I believe the next step would be to remove jupyterlab as a build dependency and replace it with jupyter-builder in template's pyproject.toml. Could I get your thoughts on this approach?

[Darshan808]

Also

I have updated the extension's pyproject.toml to include jupyter-builder as a build dependency. And, I modified all the relevant commands to use jupyter-builder in place of jupyterlab/builder.

The build process is now using jupyter-builder and is running smoothly. How should we proceed from here?

[Darshan808]

@fcollonval
I would appreciate it if you could review my message whenever you get a chance.

[fcollonval]

How should we proceed from here?

@Darshan808 would you mind opening a PR with your changes bringing in the builder TypeScript code?

Will see about the next step when the PR is merged. But yes it will mean changing some configurations in other repositories; focusing first on JupyterLab core.