Add parquet support for importing and exporting data to/from DataFrame.

[andrei-faber]

At the current moment, the only supported way to import/export data is to use CSV format; it would be very useful to add support for some more commonly used data formats or databases.

[michaelgsharp]

What other data formats are you thinking of? If we can get specific formats people want it helps with prioritizing things.

[andrei-faber]

@michaelcfanning I think that Parquet would be useful too, not sure about other formats.

[michaelcfanning]

I think you meant @michaelgsharp but, since I'm here, I agree Parquet would be useful. :)

[andrei-faber]

@michaelcfanning ah sorry my bad :)

[michaelgsharp]

I'll go ahead and change this title to be adding parquet support then.

Glad you could join us, other Michael :)

[adamreeve]

Hi, I'm one of the maintainers of ParquetSharp, and we've just released the first version of ParquetSharp.DataFrame which supports reading Parquet into DataFrames and exporting DataFrames to Parquet. The API isn't yet stable but functionality should be fairly complete so we'd love to get some feedback.

Microsoft may still want to provide built-in Parquet support in Microsoft.Data.Analysis so this probably doesn't fix this issue, but is hopefully useful to anyone following this issue, and if our work could be useful for providing that built-in Parquet support we'd be happy to help out where we can.

[michaelgsharp]

@adamreeve thanks for pointing this out!

We are currently in the process of creating/prioritizing our dataprep plan so we will definitely keep this in mind.

[pgovind]

I've been out of the loop here for a while, so this may not be as accurate anymore: FWIW, at one point the main thought here was to create a separate DataFrame.IO library that could incrementally add support for the different data formats in common use. Just something to consider.