Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse python sources for stdlib list all the way back to 2.3 #1

Merged
merged 8 commits into from
Mar 31, 2021

Conversation

thatch
Copy link
Contributor

@thatch thatch commented Mar 29, 2021

This uses the cpython source tarballs, extracts them, and analyzes the
files and C source contained.

The C parsing is by regex, which will cause problems someday, but the
end results is still more accurate for what the union of all python
platforms might provide in terms of stdlib names than anything else out
there.
stdlibs/__init__.py Outdated Show resolved Hide resolved
@amyreese
Copy link
Member

Also, it looks like the generated py* modules aren't included?

@thatch thatch force-pushed the main branch 2 times, most recently from f58642a to d990d7b Compare March 29, 2021 15:11
@thatch
Copy link
Contributor Author

thatch commented Mar 29, 2021

I still think we should resolve the offline discussion about these being "modules" -- "top-level names" is technically correct, but both wordy and I don't think most people would understand unless they're involved in packaging.

Here's the current diff vs 3.10 (stdlibs.old is prior to this PR; I don't have a copy of 3.10 handy on this machine).

>>> stdlibs.old.module_names  - stdlibs.py310.module_names
frozenset()
>>> stdlibs.py310.module_names - stdlibs.old.module_names
frozenset({'_ctypes_test', 'xxsubtype', '_testmultiphase_nonascii_latin', '_testbuffer', '_scproxy', 'xx', '_xxsubinterpreters', '__phello__', '_overlapped', '_testimportmultiple', '_testmultiphase', 'xxlimited_35', '_testconsole', '_xxtestfuzz', '_testcapi', '_testinternalcapi', 'xxlimited'})

The names _ctypes_test, _testcapi and __phello__ are importable on my Arch install of 3.8. If they're not considered "stdlib" I'm very confused.

Copy link
Member

@amyreese amyreese left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs need updating, and fetch needs a copyright header, but otherwise I'm 👍 on this. If you want to squash or whatever, go for it and let me know when you're ready for a merge.

README.md Outdated Show resolved Hide resolved
stdlibs/__init__.py Outdated Show resolved Hide resolved
stdlibs/fetch.py Show resolved Hide resolved
Lots of other minor cleanups, I'm happy with the API now.
@thatch thatch force-pushed the main branch 2 times, most recently from 7322fa4 to 11f8afd Compare March 30, 2021 14:23
@amyreese amyreese merged commit 6278035 into omnilib:main Mar 31, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants