Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DOC: Add column name metadata to spec #17769

Merged
merged 2 commits into from
Oct 4, 2017
Merged

DOC: Add column name metadata to spec #17769

merged 2 commits into from
Oct 4, 2017

Conversation

cpcloud
Copy link
Member

@cpcloud cpcloud commented Oct 3, 2017

This adds an additional metadata field to the spec to allow faithful
reproduction of names of column indexes.

cc @wesm @martindurant @jreback

@cpcloud cpcloud self-assigned this Oct 3, 2017
@cpcloud cpcloud requested a review from wesm October 3, 2017 19:44
@cpcloud cpcloud added the Docs label Oct 3, 2017
@cpcloud cpcloud added this to the 0.21.0 milestone Oct 3, 2017
@cpcloud
Copy link
Member Author

cpcloud commented Oct 3, 2017

xref: https://issues.apache.org/jira/browse/ARROW-1586

jreback
jreback reviewed Oct 3, 2017
View reviewed changes
doc/source/developer.rst
@@ -45,6 +45,7 @@ So that a ``pandas.DataFrame`` can be faithfully reconstructed, we store a
.. code-block:: text

{'index_columns': ['__index_level_0__', '__index_level_1__', ...],
'column_index_names': [<column index level name 0>, <column index level name 1>, ...],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should also store: columns_index_dtypes to faithfully restore the dtypes as well

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this one to?

@codecov
Copy link

codecov bot commented Oct 3, 2017 •
edited
Loading

Codecov Report

Merging #17769 into master will decrease coverage by 0.01%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #17769      +/-   ##
==========================================
- Coverage   91.26%   91.24%   -0.02%     
==========================================
  Files         163      163              
  Lines       49869    49869              
==========================================
- Hits        45511    45502       -9     
- Misses       4358     4367       +9
Flag Coverage Δ
#multiple 89.04% <ø> (ø) ⬆️
#single 40.24% <ø> (-0.07%) ⬇️
Impacted Files Coverage Δ
pandas/io/gbq.py 25% <0%> (-58.34%) ⬇️
pandas/core/frame.py 97.73% <0%> (-0.1%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 50c1dda...b49ff80. Read the comment docs.

@codecov
Copy link

codecov bot commented Oct 3, 2017 •
edited
Loading

Codecov Report

Merging #17769 into master will decrease coverage by 0.03%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #17769      +/-   ##
==========================================
- Coverage   91.26%   91.22%   -0.04%     
==========================================
  Files         163      163              
  Lines       49869    49916      +47     
==========================================
+ Hits        45511    45535      +24     
- Misses       4358     4381      +23
Flag Coverage Δ
#multiple 89.02% <ø> (-0.02%) ⬇️
#single 40.24% <ø> (-0.07%) ⬇️
Impacted Files Coverage Δ
pandas/io/gbq.py 25% <0%> (-58.34%) ⬇️
pandas/core/indexing.py 93% <0%> (-0.98%) ⬇️
pandas/core/frame.py 97.73% <0%> (-0.1%) ⬇️
pandas/core/generic.py 92.1% <0%> (-0.03%) ⬇️
pandas/core/common.py 91.42% <0%> (ø) ⬆️
pandas/io/formats/format.py 96.07% <0%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 50c1dda...4efb7fa. Read the comment docs.

@jreback jreback merged commit d9a8014 into pandas-dev:master Oct 4, 2017
@jreback
Copy link
Contributor

jreback commented Oct 4, 2017

thanks @cpcloud

wesm
wesm reviewed Oct 5, 2017
View reviewed changes
doc/source/developer.rst
@@ -106,6 +108,8 @@ As an example of fully-formed metadata:
.. code-block:: text

{'index_columns': ['__index_level_0__'],
'column_index_names': [None],
'column_index_dtypes': ['object'],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may want to enrich these types to include both the pandas type and the numpy type

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, I'm going to follow up here.

@cpcloud cpcloud deleted the metadata-update branch October 5, 2017 16:47
ghost pushed a commit to reef-technologies/pandas that referenced this pull request Oct 16, 2017
@cpcloud
DOC: Add column name metadata to spec (pandas-dev#17769)
1467b37
alanbato pushed a commit to alanbato/pandas that referenced this pull request Nov 10, 2017
@cpcloud @alanbato
DOC: Add column name metadata to spec (pandas-dev#17769)
f83a920
No-Stream pushed a commit to No-Stream/pandas that referenced this pull request Nov 28, 2017
@cpcloud @No-Stream
DOC: Add column name metadata to spec (pandas-dev#17769)
7dbc05b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jreback jreback jreback left review comments

@wesm wesm wesm left review comments

Assignees

@cpcloud cpcloud

Labels
Docs
Projects
None yet
Milestone
0.21.0
Development

Successfully merging this pull request may close these issues.
4 participants
@cpcloud @jreback @wesm @martindurant