Poetry groups and build-project

[randomgeek78]

Our main pyproject.toml has several groups. For each deployment, we create different folders in the projects/ folder with specialized dependencies that are needed only for that project. This setup works well.

The problem is that for each project, there are many flavors and different environments it needs to work in. Getting all our projects working on these cross-product of flavors and environments is resulting in many project directories but more importantly a lot of manual editing. Adding a new dependency needs to update the pyproject.toml files in all the associated flavor-environment project folders.

I am not sure if this is something you have encountered or solved for. If so, we would love to learn how you solved it.

One thought is to have --with to pick up groups in build-project. I understand it is still not a viable solution since it changes what it means to be a poetry project - the build name comes from the name in pyproject.toml so how will you different between which --with commands were used to create the build, etc.

I would love to hear your thoughts. Thanks!

[DavidVujic]

I think it sounds like a really interesting setup that would be cool to learn more about. I don't think I have done something like that before, but maybe a custom script would help out?

If I understand it corrrectly, the dependencies are defined at the root pyproject.toml as dependency groups. Would it be possible for you to write shell script that grabs the content of a group and run poetry add <the-dependency> --directory projects/<the-project> for each group? That script would need to be run before the build-project command.

[DavidVujic]

In the Polylith tool, there is the poly check command that will report on missing dependencies. I'm also working on something to report on dependency version mismatch between a project and the root pyproject.toml.

Example:
if a brick would import something that isn't defined in the project, the check command will print a message and also return a fail exit code (1).

from pydantic import Base

poetry poly check

🤔 Cannot locate pydantic in the_project

But there's nothing yet in the tool to synchronize the dependencies, mainly because Python dependencies/libraries are an advanced topic with many variations in naming and versioning. 😄

[DavidVujic]

(transfering this to the discussions section)