BUG: `NotImplementedError |S8` when converting categories from object containing byte arrays to bytes

[smarie]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

# Imagine we receive byte array like this but much larger (e.g. 10M rows of 128 bit uuids in bytes)
# Unfortunately not already converted to bytes dtype for some reason
a = pd.Series([b"fdhijklm", b"fdhijklm", b"fdaijklm"], dtype=object)

# We want to first convert to categorical for efficiency reasons (it is faster to first detect unique values and then convert them)
b = a.astype("category")

# Then we would like to convert the categories to proper numpy byte array for memory reasons
new_cats = b.cat.categories.astype(bytes)  # <-- NotImplementedError: |S8

Issue Description

See comments in the example

Expected Behavior

No error: new_cats would be created. Hopefully that would allow to convert the previous categories into the new ones, so that the categories take less space in memory.

Installed Versions

INSTALLED VERSIONS

commit : bdc79c1
python : 3.9.13.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.19045
machine : AMD64
processor : AMD64 Family 25 Model 68 Stepping 1, AuthenticAMD
byteorder : little
LC_ALL : None
LANG : None
LOCALE : fr_FR.cp1252
pandas : 2.2.1
numpy : 1.26.4
pytz : 2024.1
dateutil : 2.8.2
setuptools : 67.1.0
pip : 23.3.2
Cython : None
pytest : None
hypothesis : None
sphinx : 6.1.2
blosc : None
feather : None
xlsxwriter : None
lxml.etree : 5.1.0
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 3.1.3
IPython : 8.18.1
pandas_datareader : None
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : 4.12.3
bottleneck : None
dataframe-api-compat : None
fastparquet : 2024.2.0
fsspec : 2024.2.0
gcsfs : None
matplotlib : 3.8.3
numba : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : 15.0.0
pyreadstat : None
python-calamine : None
pyxlsb : None
s3fs : None
scipy : 1.12.0
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
zstandard : None
tzdata : 2024.1
qtpy : None
pyqt5 : None