Preloading the Jupyter Notebook with external libraries

[cnchann]

Is there a way to preload external libraries, such as %use lets-plot, before the start of every notebook/lab session similar to the startup feature in iPython, or are there any ways I could help in contributing to this feature?

[altavir]

Could you explain why do you need it? The dependencies are cached in the local maven repository, so they are effectively pre-loaded after the first run of the notebook.

[cnchann]

I am introducing a custom library using the %use command but it takes a long time to load. I wish to speed up the load time, are there any ways you would suggest going about it?

[ileasile]

There is no such way for now

[altavir]

@cnchann could you check why it takes a long time to load? As I said previously, if you have a large JAR, then you will have to wait for the first time until it is downloaded to the maven cache. The subsequent launches should be fast. If you have large resources (like JS bundle), then it could really take a lot of time to pre-load them in the notebook.

[cnchann]

@altavir i think it is because of a large JAR file, is there a way to preload it on maven cache, so it would be fast even on the first run? another question i would like to ask is if the maven cache is shared amongst all the notebooks in the jupyterHub or each notebook will have its own cache? thanks

[altavir]

@cnchann Maven cache is shared and usually located in ~/.m2 directory. Unlike npm or pip/conda environments, it stores different versions of the same artifact so you can use different versions in different notebooks. The simplest way to load your jars into the cache is to run a notebook with a single use directive.

I am not sure which dependency resolver is bundled in kotlin-scripting right now, but if it is updated to maven, then you can create a kts script include @DependsOn loads there and run it by kotlin command-line compiler.

Otherwise, I believe it is possible to install the maven command-line tool, create appropriate pom and do something along these lines: https://stackoverflow.com/questions/35944964/pre-download-all-dependencies. I am not sure that there is a simple solution using kotlin kernel own facilities.