Skip to content

BUG: series.replace() fails when string series contains pd.NA #43344

Closed
@stevendavis

Description

@stevendavis
  • [x ] I have checked that this issue has not already been reported.

  • [x ] I have confirmed this bug exists on the latest version of pandas.

  • (optional) I have confirmed this bug exists on the master branch of pandas.


Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Code Sample, a copy-pastable example

def test_pandas_replace_na():
    series = pd.Series(["AA", "BB", "CC", "DD", "EE", "", pd.NA], dtype="string")
    original = list(series)
    regex_mapping = {
        "AA": "CC",
        "BB": "CC",
        "EE": "CC",
        "CC": "CC-REPL",
        }
    series2 = series.replace(regex_mapping, regex=True)
    assert list(series2) == ["CC", "CC", "CC-REPL", "DD", "CC", "", pd.NA]

Problem description

The test case above causes this error in compare_or_regex_search():
ValueError: NumPy boolean array indexing assignment cannot assign 7 input values to the 6 output values where the mask is true

Expected Output

See test case.
Is there a suggested workaround?

Output of pd.show_versions()

INSTALLED VERSIONS

commit : 5f648bf
python : 3.8.8.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.18362
machine : AMD64
processor : Intel64 Family 6 Model 85 Stepping 7, GenuineIntel
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : English_United States.1252

pandas : 1.3.2
numpy : 1.20.2
pytz : 2021.1
dateutil : 2.8.2
pip : 21.1.1
setuptools : 49.6.0.post20210108
Cython : None
pytest : 6.2.4
hypothesis : None
sphinx : 4.1.2
blosc : None
feather : None
xlsxwriter : 3.0.1
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.11.3
IPython : 7.23.0
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : 3.4.1
numexpr : None
odfpy : None
openpyxl : 3.0.7
pandas_gbq : None
pyarrow : None
pyxlsb : None
s3fs : None
scipy : 1.6.3
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : 0.53.1

Metadata

Metadata

Labels

BugStringsString extension data type and string datareplacereplace method

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions