-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: make stl.h list|set|map_caster
more user friendly.
#4686
Merged
Merged
Changes from 2 commits
Commits
Show all changes
55 commits
Select commit
Hold shift + click to select a range
d2a36d9
Add `test_pass_std_vector_int()`, `test_pass_std_set_int()` in test_stl
7d30378
Change `list_caster` to also accept generator objects (`PyGen_Check(s…
039d1b7
Drop in (currently unpublished) PyCLIF code, use in `list_caster`, ad…
bdba857
Use `PyObjectTypeIsConvertibleToStdSet()` in `set_caster`, adjust tests.
ec50ca0
Use `PyObjectTypeIsConvertibleToStdMap()` in `map_caster`, add tests.
eeb59af
Simplify `list_caster` `load()` implementation, push str/bytes check …
3631886
clang-tidy cleanup with a few extra `(... != 0)` to be more consistent.
b4f767b
Also use `PyObjectTypeIsConvertibleToStdVector()` in `array_caster`.
42e22d7
Merge branch 'master' into list_caster_pass_generator
9647a92
Merge branch 'master' into list_caster_pass_generator
1ed90cf
Merge branch 'master' into list_caster_pass_generator
3cc034b
Update comment pointing to clif/python/runtime.cc (code is unchanged).
18c4153
Merge branch 'master' into list_caster_pass_generator
e7330f2
Comprehensive test coverage, enhanced set_caster load implementation.
da29bf5
Resolve clang-tidy eror.
1fa338e
Add a long C++ comment explaining what led to the `PyObjectTypeIsConv…
6b5c780
Minor function name change in test.
9150a88
Merge branch 'master' into list_caster_pass_generator
62fda81
Merge branch 'master' into list_caster_pass_generator
4b25f74
strcmp -> std::strcmp (thanks @Skylion007 for catching this)
3cfc95b
Add `PyCallable_Check(items)` in `PyObjectTypeIsConvertibleToStdMap()`
59509f2
Resolve clang-tidy error
903faac
Merge branch 'master' into list_caster_pass_generator
4ab40d7
Use `PyMapping_Items()` instead of `src.attr("items")()`, to be inter…
5f7c9d4
Update link to PyCLIF sources.
ccfeb02
Fix typo (thanks @wangxf123456 for catching this)
6c1ec6a
Merge branch 'master' into list_caster_pass_generator
cc4d69c
Merge branch 'master' into list_caster_pass_generator
dacb827
Add `test_pass_std_vector_int()`, `test_pass_std_set_int()` in test_stl
3fc2d2a
Change `list_caster` to also accept generator objects (`PyGen_Check(s…
ff0724f
Drop in (currently unpublished) PyCLIF code, use in `list_caster`, ad…
be02291
Use `PyObjectTypeIsConvertibleToStdSet()` in `set_caster`, adjust tests.
0a79f55
Use `PyObjectTypeIsConvertibleToStdMap()` in `map_caster`, add tests.
ae95424
Simplify `list_caster` `load()` implementation, push str/bytes check …
8aa81d7
clang-tidy cleanup with a few extra `(... != 0)` to be more consistent.
4435ed0
Also use `PyObjectTypeIsConvertibleToStdVector()` in `array_caster`.
9aca5e8
Update comment pointing to clif/python/runtime.cc (code is unchanged).
9dc7d7c
Comprehensive test coverage, enhanced set_caster load implementation.
d9eeea8
Resolve clang-tidy eror.
9ac319b
Add a long C++ comment explaining what led to the `PyObjectTypeIsConv…
b834767
Minor function name change in test.
d19a0ff
strcmp -> std::strcmp (thanks @Skylion007 for catching this)
69a0da8
Add `PyCallable_Check(items)` in `PyObjectTypeIsConvertibleToStdMap()`
c86fa97
Resolve clang-tidy error
940e190
Use `PyMapping_Items()` instead of `src.attr("items")()`, to be inter…
f7725e4
Update link to PyCLIF sources.
fa72918
Fix typo (thanks @wangxf123456 for catching this)
a6b9a00
Merge branch 'list_caster_pass_generator' of https://github.com/rwgk/…
ac3ac4d
Merge branch 'master' into list_caster_pass_generator
87ff92a
Merge branch 'master' into list_caster_pass_generator
3ae34f3
Fix typo discovered by new version of codespell.
d1ffe4f
Merge branch 'master' into list_caster_pass_generator
2267e5c
Merge branch 'master' into list_caster_pass_generator
a38d0bd
Merge branch 'master' into list_caster_pass_generator
fbf41fd
Merge branch 'master' into list_caster_pass_generator
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't we really be supporting a
py::iterable
object instead? I think a generator would count here and simplify the code.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I definitely like the general thinking.
Please give me a little more time. I'm in the middle of what I am thinking of as "data driven cleanup".
PyCLIF is very far on the permissive side of the spectrum. — That's what's giving me the data: what is being used, how much.
pybind11 is very far on the strict side.
A glimpse on where I am right now: PyCLIF supports passing any iterable for a C++ set. I think that's going too far to the permissive side of the spectrum. So I'm going through cleaning that up, even though it's a lot of leg work. But I believe it makes the code safer and more readable.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just pushed the equivalent of the behavior I settled on for PyCLIF. I'm still working on rolling out the PyCLIF behavior changes internally. Only after that can I submit internally and then export the changed PyCLIF to github.
High-level: I'm converging the behaviors
That's what PyCLIF used to do, but it has pitfalls that led to lots of unclean test code: Python
{}
(empty dict) passed as input forstd::vector
andstd::set
,[]
as input forstd::set
andstd::map
.The originally very permissive PyCLIF behavior gave me a fairly unique set of data: I could see what's being used a lot, what makes sense (IMO), and what doesn't make sense or seems unsafe (again IMO). The new code block near the top of stl.h, transferred here from PyCLIF, encodes my conclusions. With that the internal cleanup turned out to be relatively small, only around 50 changes I had to mail out. This is part of the change description for those (slightly edited):
Motivations for the behavior change:
Improves safety / prevents accidents:
For C++
std::set
(andstd::unordered_set
): The potentially reducing conversion from a Python sequence (e.g. Pythonlist
ortuple
) to a C++ set must be explicit.For C++
std::vector
: In very rare cases (itertools.chain
) the conversion from a Python iterable must be explicit.Improves readability: Python literals to be passed as C++ sets must be set literals.
I'll finish the rollout of the PyCLIF change, then come back here for polishing maybe and further expanding the tests. Please let me know any high-level suggestions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The PyCLIF behavior change (with prior cleanup around the codebase) is now complete: google/clif@30c5488
Internally I'm testing (globally) PyCLIF-pybind11 with this PR. Unfortunately some unrelated problem slipped in that I need to troubleshoot first, before I can convince myself that this PR actually fixes all pybind11/stl.h vs PyCLIF behavior differences.