API: Define API for pandas plotting backends

[datapythonista]

In #26414 we splitted the pandas plotting module into a general plotting framework able to call different backends and the current matplotlib backends. The idea is that other backends can be implemented in a simpler way, and be used with a common API by pandas users.

The API defined by the current matplotlib backend includes the objects listed next, but this API can probably be simplified. Here is the list with questions/proposals:

Non-controversial methods to keep in the API (They provide the Series.plot(kind='line')... functionality):

• LinePlot
• BarPlot
• BarhPlot
• HistPlot
• BoxPlot
• KdePlot
• AreaPlot
• PiePlot
• ScatterPlot
• HexBinPlot

Plotting functions provided in pandas (e.g. pandas.plotting.andrews_curves(df))

• andrews_curves
• autocorrelation_plot
• bootstrap_plot
• lag_plot
• parallel_coordinates
• radviz
• scatter_matrix
• table

Should those be part of the API and other backends should also implement them? Would it make sense to convert to the format .plot (e.g. DataFrame.plot(kind='autocorrelation')...)? Does it make sense to keep out of the API, or move to a third-party module?

Redundant methods that can possibly be removed:

• hist_series
• hist_frame
• boxplot
• boxplot_frame
• boxplot_frame_groupby

In the case of boxplot, we currently have several ways of generating a plot (calling mainly the same code):

1. DataFrame.plot.boxplot()
2. DataFrame.plot(kind='box')
3. DataFrame.boxplot()
4. pandas.plotting.boxplot(df)

Personally, I'd deprecate number 4, and for number 3, deprecate or at least not require a separate boxplot_frame method in the backend, but try to reuse BoxPlot (for number 3 comments, same applies to hist).

For boxplot_frame_groupby, didn't check in detail, but not sure if BoxPlot could be reused for this?

Functions to register converters:

• register
• deregister

Do those make sense for other backends?

Deprecated in pandas 0.23, to be removed:

• tsplot

To see what each of these functions do in practise, it may be useful this notebook by @liirusuk: https://github.com/python-sprints/pandas_plotting_library/blob/master/AllPlottingExamples.ipynb

CC: @pandas-dev/pandas-core @tacaswell, @jakevdp, @philippjfr, @PatrikHlobil

[TomAugspurger]

I think keep things like autocorrelation out of the swappable backend API.

I think we’ve left things like df.boxplot and hist around because they have slightly different behavior than the .plot API. I wouldn’t recommend making them part of the backend API.

[TomAugspurger]

Here’s my start on a proposed backend API from a few months ago: TomAugspurger@b07aba2

[datapythonista]

I think it's worth mentioning that at least hvplot (didn't check the rest) does already provide the functions like andrews_curves, scatter_matrix, lag_plot,...

May be if we don't want to force all backends to implement those, we can check if the selected backend implements them, and default to the matplotlib plots?

I assumed boxplot and hist behaved exactly the same, but just had shortcuts Series.hist() for Series.plot.hist(). The "shortcut" shows the plot grid, but other than that I haven't seen any difference.

[TomAugspurger]

IMO, the main value of this option is the `.plot` namespace. If users want hvplot's Andrew's curve plot, they should import the function from hvplot and pass the dataframe there.

…

On Sun, Jun 9, 2019 at 7:17 AM Marc Garcia ***@***.***> wrote: I think it's worth mentioning that at least hvplot (didn't check the rest) does already provide the functions like andrews_curves, scatter_matrix, lag_plot,... May be if we don't want to force all backends to implement those, we can check if the selected backend implements them, and default to the matplotlib plots? I assumed boxplot and hist behaved exactly the same, but just had shortcuts Series.hist() for Series.plot.hist(). The "shortcut" shows the plot grid, but other than that I haven't seen any difference. — You are receiving this because you are on a team that was mentioned. Reply to this email directly, view it on GitHub <#26747?email_source=notifications&email_token=AAKAOIRLJHBMXMXKK2IG2NDPZTYFPA5CNFSM4HWIMEK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXII77Y#issuecomment-500207615>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAKAOISDHL6H7PVOOJAQXELPZTYFPANCNFSM4HWIMEKQ> .

[datapythonista]

I think that makes sense, but if we do that, I think we should move them to pandas.plotting.matplotlib.andrews_curves, instead of pandas.plotting.andrews_curves.

@TomAugspurger I need to check in more detail, but I think the API you implemented in TomAugspurger@b07aba2 is the one that makes more sense. I'll work on it once I finish #26753. I'll also experiment on whether it's feasible to move andrews_curves, scatter_matrix... to the .plot() syntax, I think that will make things simpler and easier for everyone (us, third-party libraries, and users).

[jakevdp]

What's the intention here regarding extra kwargs passed to plotting functions? Should additional backends attempt to duplicate the functionality of all matplotlib-style plot customizations, or should they allow keywords to be passed that correspond to those used by the particular backend?

The first option would be nice in theory, but would require every non-matplotlib plotting backend to essentially implement its own matplotlib conversion layer with a long tail of incompatibilities that would essentially never be complete (speaking from experience as someone who tried to create mpld3 some years back).

The second option is not as nice from the perspective of interchangeability, but would allow other backends to be added with a more reasonable set of expectations.

[TomAugspurger]

I think that's up to the backend on what they do with them. Achieving 100% compatibility across backends isn't really feasible, since the return type isn't going to be a matplotlib Axes anymore. And if we aren't compatible on the return type, I don't think backends should bend over backwards to try to handle every possible keyword argument. So I think pandas should document that `**kwargs` will be passed through to the underlying plotting engine, and they can do whatever they please with them.

…

On Mon, Jun 10, 2019 at 10:42 AM Jake Vanderplas ***@***.***> wrote: What's the intention here regarding extra kwargs passed to plotting functions? Should additional backends attempt to duplicate the functionality of all matplotlib-style plot customizations, or should they allow keywords to be passed that correspond to those used by the particular backend? The first option would be nice in theory, but would require every non-matplotlib plotting backend to essentially implement its own matplotlib conversion layer with a long tail of incompatibilities that would essentially never be complete (speaking from experience as someone who tried to create mpld3 some years back). The second option is not as nice from the perspective of interchangeability, but would allow other backends to be added with a more reasonable set of expectations. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#26747?email_source=notifications&email_token=AAKAOIS3IBV4XSSY7BPSCF3PZZY5LA5CNFSM4HWIMEK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXKH4AY#issuecomment-500465155>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAKAOIQ3GYOGAPUZ4LSNK2DPZZY5LANCNFSM4HWIMEKQ> .

I'm sorry if this is a stupid question, but If you define a plotting "API" which is basically a group of canned plots, wouldn't every backend produce more or less the same output? what new capability is this meant to enable? something like a pandas to vega exporter perhaps?

[jakevdp]

I don't think it's correct to say that every backend produces more or less the same output.

For example, matplotlib is really good at static charts, but not great at producing portable interactive charts.

On the other hand, bokeh, altair, et al. are great for interactive charts, but aren't quite as mature as matplotlib for static charts.

Being able to produce both with the same API would be a big win.

[tacaswell]

The first option would be nice in theory, but would require every non-matplotlib plotting backend to essentially implement its own matplotlib conversion layer with a long tail of incompatibilities that would essentially never be complete (speaking from experience as someone who tried to create mpld3 some years back).

and also pins Matplotlib down even more than we already are API wise. I think it makes sense for pandas to declare what style knobs it wants to expose and expect the backend implementations to sort out what that means. This may mean not blindly passing **kwargs through and instead ensuring that the returned objects are "the right thing" for the given backend to be able to do after-the-fact style customization.