Use Hatch Environments and Scripts

[RobertRosca]

PR implements some of the Hatch features mentioned in #209, main ones are:

• Matrix definitions for incompatible plugins
  • The dev environment is a matrix currently made up of the all combinations of frontends ['lab', 'retro'] and authentication plugins ['noauth', 'auth', 'fief]
  • This lets you have all plugins installed in their own environments, and allows for easy testing/switching between them
  • hatch run dev.lab-noauth:jupyverse spawns jupyverse with jupyterlab with noauth, hatch run dev.retro-fief:jupyverse is retro with fief auth, etc...
• Docs environment and scripts allow for easy testing/building of docs, e.g. hatch run docs:serve

Docs were updated to explain the above.

Right now this isn't ideal as using context formatting to specify the plugins as dependencies for development didn't work, since Hatch doesn't have an easy mechanism for creating environments with editable dependency installations. So for now this is done with pre/post install commands running pip install on the required plugins.

I've started a discussion on the Hatch repo here pypa/hatch#516 to see if a PR allowing for the easy definition of editable local dependencies would be accepted. If it is then I'd like to wait for that. and remove the workaround calling pip install in the pre/post script sections.

Still to do:

• Wait for new Hatch release with Context Formatting Support for Environments pypa/hatch#484 merged
• Update GitHub Workflows to use hatch environments during setup
• Add Hatch template to make creation of new plugins easier/standardised

Closes #209

[davidbrochart]

Thanks a lot @RobertRosca!
I just changed the environment names, it was a bit confusing because we have a lab plugin which is common to the jupyterlab and retrolab plugins, so it's better to keep the exact names jupyterlab and retrolab for the frontend.
I couldn't have e.g. hatch run dev.jupyterlab-noauth jupyverse working, nor hatch -e dev.jupyterlab-noauth shell. It seems that it doesn't install the jupyterlab plugin. Any idea?

[RobertRosca]

Part of this is the Hatch syntax. You need a colon between the environment name and the command to be executed, like hatch run dev.jupyterlab-noauth:jupyverse.

But that also doesn't work thanks to come copy/paste fumbling from me 😛, problem was:

[tool.hatch.envs.dev.scripts]
jupyverse = "fps_uvicorn.cli:app"

...

[project.scripts]
jupyverse = "fps_uvicorn.cli:app"

This meant that the bin/jupyverse was correctly created because of the [project.script] section (which defines entry points), but hatch run {env}:jupyverse would call /bin/sh fps_uvicorn.cli:app due to the [envs.*.scripts] section (which defines commands that get called by the shell), causing the issue.

nor hatch -e dev.jupyterlab-noauth shell

I've fixed that now, could you give it another go?

Didn't notice either of these issues since I forgot to run hatch env purge and old environments were being re-used, meaning some changes weren't reflected.

[davidbrochart]

Awesome, that works!

[davidbrochart]

It looks like pypa/hatch#484 has been merged and released.
Looking forward to getting this PR in, but there is no rush @RobertRosca, whenever you find time 👍

[RobertRosca]

Hey, sorry for the delay, was busy with work last week. I'm debugging the test failures locally, not clear to me why it passes on 3.7 but not on 3.8+ yet (now it is 😛 working on a fix)

Related to that, I'm using act to run tests locally and there is a bug in the setup-miniconda action which means that it fails local executions without setting the version explicitly, which is why I have set it to latest in df2bc3d (#239).

[RobertRosca]

Sweet, I rebased off of #241 and tested locally, everything seems to be working now. Once #241 gets merged the issues here should be fixed 👍

[davidbrochart]

This looks great @RobertRosca, this will improve Jupyverse's workflow management so much!
I just tried hatch run dev.jupyterlab-noauth:jupyverse, and got:

fps_uvicorn.cli:app: not found

[RobertRosca]

Oops, screwed up when rebasing and re-introduced #239 (comment)

It should work now, could you give it another go?

[davidbrochart]

Yes, it works fine now!

[davidbrochart]

Just a minor update to docs, and I think it's good to go!
Do you want to squash a little bit, especially to get rid of 1653b7e?

[RobertRosca]

Oh yes, I'll go ahead and do that

[RobertRosca]

Could also just do a squash merge, don't think anything too important would be lost from the commit log from this

[davidbrochart]

I think you deserve more credit, thanks a lot for your contribution @RobertRosca !
It looks like tests hang on windows for python >=3.8, I'll merge and investigate later.

[ofek]

Awesome job!