-
-
Notifications
You must be signed in to change notification settings - Fork 17.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ENH: add consortium standard entrypoint #54383
ENH: add consortium standard entrypoint #54383
Conversation
No objections here. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you add this to the docs as an optional dependency?
(I assume we'll want a minimum version of the standard at some point)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good to me now.
Can you add a whatsnew too in 2.1.0 saying that pandas supports the consortium or something like that too?
Thanks @MarcoGorelli |
* add consortium standard entrypoint * add dataframe-api-compat to optional dependencies, add to test_downstream * align * use importorskip * mypy * fixup table in docs * whatsnew * add check to package-checks.yml
Here's an entry point to the Consortium's DataFrame API Standard
It enables dataframe-consuming libraries to just check for a
__dataframe_consortium_standard__
attribute on a dataframe they receive - then, so long as they stick to the spec defined in https://data-apis.org/dataframe-api/draft/index.html, then their code should work the same way, regardless of what the original backing dataframe library wasUse-case: currently, scikit-learn is very keen on using this, as they're not keen on having to depend on pyarrow (which will become required in pandas), and the interchange protocol only goes so far (e.g. a way to convert to ndarray is out-of-scope for that). If we can get this to work there, then other use cases may emerge
The current spec should be enough for scikit-learn, and having this entry point makes it easier to move forwards with development (without monkey-patching / special-casing)
For reference, polars has already merged this: pola-rs/polars#10244
Maintenance burden on pandas
None
I want to be very clear about this: the compat package will be developed, maintained, and tested by the consortium and community of libraries which use it. It is up to consuming libraries to set a minimum version of the dataframe-api-compat package. No responsibility will land on pandas maintainers. If any bugs are reported to pandas, then they can (and should) be politely closed.
All this is just an entry point to the Consortium's Standard
Tagging @pandas-dev/pandas-core for visibility. Some people raised objections (maintenance burden, naming) when asked in private, and I think I've addressed the concerns. Anything else?
Thanks 🙌