[feat] having a uri pip or uv will understand for hub deps

[cartazio]

Description
I want to actually track guardrails hub deps as normal python deps. This would radically simplify the devX/ UX of a complex python application.

Why is this needed
No one should need two package managers to install stuff for a single language code base. And there’s no way the second one will have feature parity with the main/dedicated one.

Implementation details
I can help if needed. I guess. But this seems like a pretty important feature for making the guardrails toolchain non invasive for complex projects or build systems.

End result
I should be able to get pyproject friendly uris that pin a validator version (or track latest) that I can add to pyproject toml deps field. For reproducible builds being able to pin versions is required. Idk if the uri schemes suport that info being surfaced.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 14 days.

[kristoph4822]

Bump, I want to install Guardrails validators alongside my dependencies in pyproject.toml. For remote server I can install it separately, however for local development it means handling two package managers. Otherwise, every sync removes dependencies installed with guardrails hub install.

[alexey-asl]

@CalebCourier , perhaps you could comment on that issue?

An attempt to add the hub validators with uv package manager fails with the following issue,
since guardrails-grhub-profanity-free not in the standard pypi index:

uv add guardrails-grhub-profanity-free==0.0.0
  × No solution found when resolving dependencies for split (python_full_version >= '3.13'):
  ╰─▶ Because guardrails-grhub-profanity-free was not found in the package registry and your project depends on guardrails-grhub-profanity-free==0.0.0, we can conclude that your project's requirements
      are unsatisfiable.
  help: If you want to add the package regardless of the failed resolution, provide the `--frozen` flag to skip locking and syncing.

And when try to install validator again with guardrails CLI, I get the following:

.venv/lib/python3.12/site-packages/guardrails/hub/__init__.py", line 5, in <module>
    from guardrails_grhub_profanity_free import ProfanityFree
ModuleNotFoundError: No module named 'guardrails_grhub_profanity_free'

And in order to reinstall validator after uv add uv remove uv sync etc., I would need to comment out the import in guardrails/hub/__init__.py module, and then run guardrails hub install, which seems odd 🤷

If there's a way to let the package managers know about these transient hub dependencies(validators),
this at least would not cause conflicts between guardrails cli tool and python package managers such as uv

[CalebCourier]

@cartazio sorry for the delayed response. I meant to comment on this earlier. CC: @kristoph4822 and @alexey-asl for this update as well.

To comment on this as whole, the first thing to note is that hub validators are not just code. Many of them also have ML models that back them. The reason the guardrails cli exists at all (wrt to installing from the hub) is because there are post install actions that need to occur to make the validators more usable (i.e. downloading backing models & weights, setting up barrel files for easier imports, etc.). Some of this, like downloading model weights, can and will happen at runtime if post install is not run, but this takes a non-trivial amount of time hence the install-time hook instead.

There are ways to install the underlying pip packages themselves using only pip, but this will not run the post-install. One workaround is to install the pip packages first, then run the hub install command afterwards. As long as you don't pass the update flag to the hub install, it will see that the packages are already installed and only run post-install. This still requires an API key because the packages themselves are in a private pypi registry that requires authentication.

If you're interested in trying this workaround, see the code sample here: https://github.com/CalebCourier/gr-hub-pip-install

As you see in the sample repo, this works when using pip with a pyproject.toml. I have not tried this with poetry, uv, or any of the other third party package manages, but if they are truly pip compliant then they should either work with this method out-of-the-box, or have appropriate stand ins for feature parity.

[cartazio]

Is there no way to register extra pre/post install action hooks for pip packages? It seems like the whole problem ultimately comes from that gap in packaging tools.

To be clear, currently this issue doesn’t matter for work I’m doing, but it is a dev ex ouchie either way

[CalebCourier]

No, pip does not currently support pre or post install actions like some other package managers do.