Skip to content

p4a's overtaking of dependency order in place of --no-deps is problematic and IMHO should go #1490

Closed
@ghost

Description

p4a's overtaking of dependency order of pure python modules in place of --no-deps is problematic and IMHO should go, even though this will break certain builds and recipes I assume.

Let me explain why I think it should go away:

  • the CythonRecipe install process is fundamentally hacky, by not using the proper setup.py but instead manually Cythonizing with 1. wrong venv, 2. wrong python, 3. wrong site-packages, which can work or can badly break things (e.g. for projects with .pxd include dependencies)
  • therefore, I attempted to make CythonRecipe unnecessary by fixing the build setup to work out of the box (see pull request Make Cython work without recipe #1483 ), allowing the regular setup.py to run instead of the limited bypassed Cythonize of p4a
  • however, the dependency graph resolving --requirements relies on the specified dependencies in the recipes to order them, and just completely jumbles up the order of non-recipe'd dependencies. However, this order is vital for projects with .pxd dependencies, and while it's probably not commonly relevant for most others, it can be and so this is just a fundamentally problematic assumption IMHO
  • but to make the dependency graph do this properly, it would need to actually somehow query the setup.py properly to find it the order of pure python modules, since there are no recipes for them to get the deps from. This is of course not an easy task and probably a bad idea due to the complexity

The obvious alternate solution is to remove --no-deps in build.py's run_pymodules_install() func. However, this has a few implications:

  1. recipes would need to be installed properly (in a way that pip recognizes they're installed) if they're dependency of any pure python module, or things will break (pip will attempt to install them twice)
  2. all dependencies of any pure python module would always be installed with no way to stop this from happening. to change this, one would need to get a custom recipe to prevent this, while previously, one could just omit these dependencies from --requirements

However, it would have the following bonuses:

  1. the flawed dependency graph will be completely avoided for pure python modules, instead of making it even more complicated and error-prone in a possibly futile attempt to fix the ordering of pure python installs
  2. it will no longer be necessary to specify dependencies-of-dependencies for pure python modules in the --requirements list, making it easier to put together all needed requirements
  3. In general, this will move p4a a good bit further towards less recipes, less custom dependency hacking, less setup.py bypassing and just letting things install as they were designed to install

I wonder if there is a better solution, so I'm interested in input on this.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions