Scatter requires x column to be numeric

[naught101]

In [30]: df = pd.DataFrame(dict(a=['A', 'B', 'C'], b=[2, 3, 4]))

In [31]: plt.scatter(df['a'], df['b'])
Out[31]: <matplotlib.collections.PathCollection at 0x7f17e17b41d0>

This works, and produces this:

On the other hand, this doesn't:

In [32]: df.plot.scatter(x='a', y='b')
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/data/documents/uni/phd/projects/FluxnetTrafficLights/scripts/plots/predictability_plots.py in <module>()                                                                                                     
----> 1 df.plot.scatter(x='a', y='b')

~/miniconda3/envs/science/lib/python3.6/site-packages/pandas/plotting/_core.py in scatter(self, x, y, s, c, **kwds)                                                                                           
   2803         axes : matplotlib.AxesSubplot or np.array of them
   2804         """
-> 2805         return self(kind='scatter', x=x, y=y, c=c, s=s, **kwds)
   2806 
   2807     def hexbin(self, x, y, C=None, reduce_C_function=None, gridsize=None,

~/miniconda3/envs/science/lib/python3.6/site-packages/pandas/plotting/_core.py in __call__(self, x, y, kind, ax, subplots, sharex, sharey, layout, figsize, use_index, title, grid, legend, style, logx, logy, loglog, xticks, yticks, xlim, ylim, rot, fontsize, colormap, table, yerr, xerr, secondary_y, sort_columns, **kwds)                                                                                           
   2625                           fontsize=fontsize, colormap=colormap, table=table,
   2626                           yerr=yerr, xerr=xerr, secondary_y=secondary_y,
-> 2627                           sort_columns=sort_columns, **kwds)
   2628     __call__.__doc__ = plot_frame.__doc__
   2629 

~/miniconda3/envs/science/lib/python3.6/site-packages/pandas/plotting/_core.py in plot_frame(data, x, y, kind, ax, subplots, sharex, sharey, layout, figsize, use_index, title, grid, legend, style, logx, logy, loglog, xticks, yticks, xlim, ylim, rot, fontsize, colormap, table, yerr, xerr, secondary_y, sort_columns, **kwds)                                                                                         
   1867                  yerr=yerr, xerr=xerr,
   1868                  secondary_y=secondary_y, sort_columns=sort_columns,
-> 1869                  **kwds)
   1870 
   1871 

~/miniconda3/envs/science/lib/python3.6/site-packages/pandas/plotting/_core.py in _plot(data, x, y, subplots, ax, kind, **kwds)
   1650         if isinstance(data, DataFrame):
   1651             plot_obj = klass(data, x=x, y=y, subplots=subplots, ax=ax,
-> 1652                              kind=kind, **kwds)
   1653         else:
   1654             raise ValueError("plot kind %r can only be used for data frames"

~/miniconda3/envs/science/lib/python3.6/site-packages/pandas/plotting/_core.py in __init__(self, data, x, y, s, c, **kwargs)
    808             # the handling of this argument later
    809             s = 20
--> 810         super(ScatterPlot, self).__init__(data, x, y, s=s, **kwargs)
    811         if is_integer(c) and not self.data.columns.holds_integer():
    812             c = self.data.columns[c]

~/miniconda3/envs/science/lib/python3.6/site-packages/pandas/plotting/_core.py in __init__(self, data, x, y, **kwargs)
    783             y = self.data.columns[y]
    784         if len(self.data[x]._get_numeric_data()) == 0:
--> 785             raise ValueError(self._kind + ' requires x column to be numeric')
    786         if len(self.data[y]._get_numeric_data()) == 0:
    787             raise ValueError(self._kind + ' requires y column to be numeric')

ValueError: scatter requires x column to be numeric

Why does pandas require x to be numeric if matplotlib doesn't?

using versions from conda, on Kubuntu 17.10:

matplotlib                2.1.0            py36hba5de38_0  
pandas                    0.20.3                   py36_0  
python                    3.6.3                h0ef2715_3  

[sinhrks]

Related to #8113. Maybe we can pass x-values as it is?

[naught101]

I tried commenting out the lines which raise the assertion: https://github.com/pandas-dev/pandas/blob/master/pandas/plotting/_core.py#L811

But then I got this:

In [7]: df.plot.scatter(x='a', y='b')
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
~/miniconda3/envs/science/lib/python3.6/site-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)                                                                                    
   2441             try:
-> 2442                 return self._engine.get_loc(key)
   2443             except KeyError:

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc (pandas/_libs/index.c:5280)()

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc (pandas/_libs/index.c:5126)()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item (pandas/_libs/hashtable.c:20523)()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item (pandas/_libs/hashtable.c:20477)()

KeyError: 'a'

During handling of the above exception, another exception occurred:

KeyError                                  Traceback (most recent call last)
<ipython-input-7-d7c052f80cc7> in <module>()
----> 1 df.plot.scatter(x='a', y='b')

~/miniconda3/envs/science/lib/python3.6/site-packages/pandas/plotting/_core.py in scatter(self, x, y, s, c, **kwds)                                                                                           
   2803         axes : matplotlib.AxesSubplot or np.array of them
   2804         """
-> 2805         return self(kind='scatter', x=x, y=y, c=c, s=s, **kwds)
   2806 
   2807     def hexbin(self, x, y, C=None, reduce_C_function=None, gridsize=None,

~/miniconda3/envs/science/lib/python3.6/site-packages/pandas/plotting/_core.py in __call__(self, x, y, kind, ax, subplots, sharex, sharey, layout, figsize, use_index, title, grid, legend, style, logx, logy, loglog, xticks, yticks, xlim, ylim, rot, fontsize, colormap, table, yerr, xerr, secondary_y, sort_columns, **kwds)                                                                                           
   2625                           fontsize=fontsize, colormap=colormap, table=table,
   2626                           yerr=yerr, xerr=xerr, secondary_y=secondary_y,
-> 2627                           sort_columns=sort_columns, **kwds)
   2628     __call__.__doc__ = plot_frame.__doc__
   2629 

~/miniconda3/envs/science/lib/python3.6/site-packages/pandas/plotting/_core.py in plot_frame(data, x, y, kind, ax, subplots, sharex, sharey, layout, figsize, use_index, title, grid, legend, style, logx, logy, loglog, xticks, yticks, xlim, ylim, rot, fontsize, colormap, table, yerr, xerr, secondary_y, sort_columns, **kwds)                                                                                         
   1867                  yerr=yerr, xerr=xerr,
   1868                  secondary_y=secondary_y, sort_columns=sort_columns,
-> 1869                  **kwds)
   1870 
   1871 

~/miniconda3/envs/science/lib/python3.6/site-packages/pandas/plotting/_core.py in _plot(data, x, y, subplots, ax, kind, **kwds)                                                                               
   1692         plot_obj = klass(data, subplots=subplots, ax=ax, kind=kind, **kwds)
   1693 
-> 1694     plot_obj.generate()
   1695     plot_obj.draw()
   1696     return plot_obj.result

~/miniconda3/envs/science/lib/python3.6/site-packages/pandas/plotting/_core.py in generate(self)
    243         self._compute_plot_data()
    244         self._setup_subplots()
--> 245         self._make_plot()
    246         self._add_table()
    247         self._make_legend()

~/miniconda3/envs/science/lib/python3.6/site-packages/pandas/plotting/_core.py in _make_plot(self)
    841         else:
    842             label = None
--> 843         scatter = ax.scatter(data[x].values, data[y].values, c=c_values,
    844                              label=label, cmap=cmap, **self.kwds)
    845         if cb:

~/miniconda3/envs/science/lib/python3.6/site-packages/pandas/core/frame.py in __getitem__(self, key)
   1962             return self._getitem_multilevel(key)
   1963         else:
-> 1964             return self._getitem_column(key)
   1965 
   1966     def _getitem_column(self, key):

~/miniconda3/envs/science/lib/python3.6/site-packages/pandas/core/frame.py in _getitem_column(self, key)                                                                                                      
   1969         # get column
   1970         if self.columns.is_unique:
-> 1971             return self._get_item_cache(key)
   1972 
   1973         # duplicate columns & possible reduce dimensionality

~/miniconda3/envs/science/lib/python3.6/site-packages/pandas/core/generic.py in _get_item_cache(self, item)                                                                                                   
   1643         res = cache.get(item)
   1644         if res is None:
-> 1645             values = self._data.get(item)
   1646             res = self._box_item_values(item, values)
   1647             cache[item] = res

~/miniconda3/envs/science/lib/python3.6/site-packages/pandas/core/internals.py in get(self, item, fastpath)                                                                                                   
   3588 
   3589             if not isnull(item):
-> 3590                 loc = self.items.get_loc(item)
   3591             else:
   3592                 indexer = np.arange(len(self.items))[isnull(self.items)]

~/miniconda3/envs/science/lib/python3.6/site-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance)
   2442                 return self._engine.get_loc(key)
   2443             except KeyError:
-> 2444                 return self._engine.get_loc(self._maybe_cast_indexer(key))
   2445 
   2446         indexer = self.get_indexer([key], method=method, tolerance=tolerance)

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc (pandas/_libs/index.c:5280)()

pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc (pandas/_libs/index.c:5126)()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item (pandas/_libs/hashtable.c:20523)()

pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item (pandas/_libs/hashtable.c:20477)()

KeyError: 'a'

The non-numeric columns seem to get dropped somehow?

[bobhaffner]

Adding to this. Looks like plt.scatter(df['a'], df['b']) won't work with older versions of matplotlib as I get ValueError: could not convert string to float: 'C' with version '2.0.2'

[bobhaffner]

and @naught101 yeah, 'a' is getting dropped here

[jadolfbr]

Any way around this issue still using the plot function?

[ckaldemeyer]

I have the same error and it occurs only when there is a column of type DateTime which is not even selected for the scatter plot.

[RahulRAhhy123]

I am also getting a same error ValueError: scatter requires x column to be numeric when I am trying plot a column which is of datetime type...how to resolve this @naught101 @ckaldemeyer @sinhrks @bobhaffner @jadolfbr

[jadolfbr]

@RahulRAhhy123 - I don't have a workaround for this. I wish I could use scatter directly from pandas with a jitter for the values. For some types of data, this is much clearer than a bar plot.

[pietz]

What I did when I tried to plot a Date feature over multiple years on the x-axis was to convert it to a year-float representation like so:

df['DateNum'] = df.Date.dt.year + df.Date.dt.month / 12. + df.Date.dt.day / 30.

2018-09-01 --> 2018.75

This isn't super accurate, nor is it pretty but it got the job done for me.

[fmagin]

I am running into this issue when trying to scatterplot with x and y both initially being lists of datetime objects, but I have a workaround using matplotlib directly:

df = pd.DataFrame({'x':x,'y':y})

# Fails with 'ValueError: scatter requires x column to be numeric'
df.plot.scatter(x='x', y='y')

# Works, xdate=True is implicit
plt.plot_date(df['x'], df['y'], ydate=True)

[jasonbono]

Some of the workarounds may work, but it's an inconvenient bug that I'm, and presumably others, are still encountering. Does anyone know if the actual bug is being address?

[TomAugspurger]

I don't think anyone is working on this right now. Are you interested @jasonbono?