Fix PandasArrayExtensionArray.take coercing fill_value when not filling - #8486
Draft
aryansk wants to merge 1 commit into
Draft
Fix PandasArrayExtensionArray.take coercing fill_value when not filling#8486aryansk wants to merge 1 commit into
aryansk wants to merge 1 commit into
Conversation
fill_value was coerced to the pandas dtype before checking whether any row was actually missing, so calling take() with a complete mask and a fill_value that does not fit the dtype (e.g. False for a str column) raised a conversion error. Coerce only when filling is actually needed, and always return a proper pandas Series.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #8465
Problem
PandasArrayExtensionArray.takecoercesfill_valueto the pandas dtype before checking whether any row is actually missing. Whenallow_fill=Truewith a mask that contains no missing indices,takestill needs to return a proper pandas Series — but the coercion can blow up for values that don't fit the column dtype, e.g.:Change
Coerce
fill_valueto the pandas dtype only when filling is actually needed (a missing index was requested), and always return a properpandas.Seriesfromtake— previously the no-fill path could return a barenp.ndarray, breaking downstream code that expectsSeriessemantics (indexing by boolean mask etc.).Testing
takewithallow_fill=Trueand a complete mask no longer errors on a non-matchingfill_value, and the result matches a plaintake.