Update arrow to include pandas serialization #1102
Conversation
python/ray/worker.py
Outdated
| return { | ||
| 'type': 'Series', | ||
| 'data': serialized.to_pybytes() | ||
| } |
There was a problem hiding this comment.
indentation is off here
python/ray/worker.py
Outdated
| 'data': serialized.to_pybytes() | ||
| } | ||
|
|
||
|
|
There was a problem hiding this comment.
too much white space between these functions, should be one line, run flake8 worker.py.
pcmoritz
force-pushed
the
pandas-serialization
branch
from
27442cc to
558bd1d
Compare
|
Merged build finished. Test PASSed. |
|
Test PASSed. |
|
Merged build finished. Test PASSed. |
|
Test PASSed. |
|
Merged build finished. Test PASSed. |
|
Test PASSed. |
|
We could also just use the pyarrow default context. And we could add our defaultdict and OrderedDict custom serializers to the default context. What do you think about that? |
|
Yeah, I'm planning on doing that and putting everything into the default context in arrow and getting rid of most of our code; this will require another PR to arrow. |
No description provided.