BUG: histogram weights aren't dropped if NaN values in data

[AdamOrmondroyd]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

>>> import numpy as np
>>> from pandas import DataFrame
>>> df = DataFrame(np.random.randn(100), columns=["a"])
>>> weights = np.ones(100)
>>> df.loc[0, "a"] = np.nan
>>> df.plot.hist() # works fine
>>> df.plot.hist(weights=weights) # fails

Issue Description

If a DataFrame contains NaN values, these are dropped to plot a histogram. However, if weights are provided, then the corresponding weights aren't dropped, so weights ends up longer than the plotted data:

Traceback (most recent call last):
  File "/Users/adam/Programming/multinesttest/pandas_issue.py", line 10, in <module>
    df.plot.hist(weights=weights) # fails
  File "/Users/adam/Programming/env_sep30/lib/python3.10/site-packages/pandas/plotting/_core.py", line 1375, in hist
    return self(kind="hist", by=by, bins=bins, **kwargs)
  File "/Users/adam/Programming/env_sep30/lib/python3.10/site-packages/pandas/plotting/_core.py", line 1001, in __call__
    return plot_backend.plot(data, kind=kind, **kwargs)
  File "/Users/adam/Programming/env_sep30/lib/python3.10/site-packages/pandas/plotting/_matplotlib/__init__.py", line 71, in plot
    plot_obj.generate()
  File "/Users/adam/Programming/env_sep30/lib/python3.10/site-packages/pandas/plotting/_matplotlib/core.py", line 453, in generate
    self._make_plot()
  File "/Users/adam/Programming/env_sep30/lib/python3.10/site-packages/pandas/plotting/_matplotlib/hist.py", line 154, in _make_plot
    artists = self._plot(ax, y, column_num=i, stacking_id=stacking_id, **kwds)
  File "/Users/adam/Programming/env_sep30/lib/python3.10/site-packages/pandas/plotting/_matplotlib/hist.py", line 108, in _plot
    n, bins, patches = ax.hist(y, bins=bins, bottom=bottom, **kwds)
  File "/Users/adam/Programming/env_sep30/lib/python3.10/site-packages/matplotlib/__init__.py", line 1423, in inner
    return func(ax, *map(sanitize_sequence, args), **kwargs)
  File "/Users/adam/Programming/env_sep30/lib/python3.10/site-packages/matplotlib/axes/_axes.py", line 6652, in hist
    raise ValueError('weights should have the same shape as x')
ValueError: weights should have the same shape as x

Expected Behavior

The weights corresponding to the NaN data should be dropped before plotting the histogram.

I believe the problem lies here:

pandas/pandas/plotting/_matplotlib/hist.py

Lines 147 to 156 in 71c94c3

	y = reformat_hist_y_given_by(y, self.by)
	# We allow weights to be a multi-dimensional array, e.g. a (10, 2) array,
	# and each sub-array (10,) will be called in each iteration. If users only
	# provide 1D array, we assume the same weights is used for all iterations
	weights = kwds.get("weights", None)
	if weights is not None and np.ndim(weights) != 1:
	kwds["weights"] = weights[:, i]
	artists = self._plot(ax, y, column_num=i, stacking_id=stacking_id, **kwds)

I think the order should be: weights is popped, drop the weights corresponding to the NaN values of y, before y = reformat_hist_y_given_by(y, self.by).

Installed Versions

Replace this line with the output of pd.show_versions()

INSTALLED VERSIONS

---

commit : 87cfe4e
python : 3.10.6.final.0
python-bits : 64
OS : Darwin
OS-release : 21.6.0
Version : Darwin Kernel Version 21.6.0: Mon Aug 22 20:19:52 PDT 2022; root:xnu-8020.140.49~2/RELEASE_ARM64_T6000
machine : arm64
processor : arm
byteorder : little
LC_ALL : None
LANG : en_GB.UTF-8
LOCALE : en_GB.UTF-8

pandas : 1.5.0
numpy : 1.23.3
pytz : 2022.2.1
dateutil : 2.8.2
setuptools : 63.4.3
pip : 22.2.2
Cython : None
pytest : 7.1.3
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
brotli : None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : 3.6.0
numba : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : 1.9.1
snappy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
zstandard : None
tzdata : None

[kostyafarber]

Hi, I'd like to take a look into this?

[kostyafarber]

take

[AdamOrmondroyd]

I've had an attempt, see #48888