ARROW-92: Arrow to Parquet Schema conversion

[xhochy]

My current WIP state. To make the actual schema conversion complete, we probably need the physical structure too as Arrow schemas only care about logical types whereas Parquet schema is about logical and physical types.

[wesm]

We'll have to make some decisions about type mappings. For example:

• arrow::StringType becomes BYTE_ARRAY with UTF8 annotation
• arrow::BinaryType (needs to be implemented) becomes BYTE_ARRAY with no ConvertedType
• arrow::CharType (if ever used, we can skip it for now) becomes FIXED_LEN_BYTE_ARRAY

For List types, we should use the 3-level array encoding as described here https://github.com/apache/parquet-cpp/blob/master/src/parquet/schema/types.h#L49

What other parts do you think are underspecified?

[xhochy]

For Decimal we need to decide if use the smallest possible physical type is the correct strategy.

[wesm]

I see. For decimals, I agree we either need multiple Arrow types, or add metadata indicating the physical storage type to the DecimalType. I would say it's better to make this explicit in the Arrow data type, let me know what you think

[xhochy]

Probably a simple storage_type field could be enough for the DecimalType. As this probably needs to go into the spec, I made separate issues for this https://issues.apache.org/jira/browse/ARROW-183 and https://issues.apache.org/jira/browse/ARROW-184

[xhochy]

PR is now in state for a minimal schema conversion basis for Pandas<->Parquet.

[wesm]

Aside: we'll need to visit the string encoding question, as logical unicode characters won't map neatly onto a char(n) type

[wesm]

This looks good outside the exception handling question

[wesm]

+1, thank you