Skip to content

Conversation

@zhijunfu
Copy link
Contributor

This change targets to improve a few places in current plasma notification code:

  1. When a client subscribes to Plasma, the store pushes notifications about existing objects to ALL subscribers, while it should only push to the new subscriber.
  2. And in the above scenario, it should only push "sealed" objects to the new subscriber, while currently it pushes all objects regardless of the state.
  3. When a client disconnects, it will no longer be able to receive notifications, thus the NotificationQueue for the client should be removed from global map.

@codecov-io
Copy link

codecov-io commented May 11, 2018

Codecov Report

Merging #2031 into master will increase coverage by 0.02%.
The diff coverage is 94.73%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2031      +/-   ##
==========================================
+ Coverage   86.35%   86.37%   +0.02%     
==========================================
  Files         242      230      -12     
  Lines       41184    40474     -710     
==========================================
- Hits        35565    34960     -605     
+ Misses       5619     5514     -105
Impacted Files Coverage Δ
cpp/src/plasma/store.h 100% <ø> (ø) ⬆️
cpp/src/plasma/store.cc 88.71% <94.73%> (-0.05%) ⬇️
python/pyarrow/array.pxi 66.04% <0%> (-2.5%) ⬇️
python/pyarrow/feather.pxi 67.44% <0%> (-1.45%) ⬇️
cpp/src/arrow/util/bit-util.h 98.01% <0%> (-1.23%) ⬇️
python/pyarrow/types.pxi 57.9% <0%> (-1.09%) ⬇️
python/pyarrow/table.pxi 68.69% <0%> (-1.07%) ⬇️
python/pyarrow/__init__.py 57.14% <0%> (-0.76%) ⬇️
python/pyarrow/io.pxi 62.23% <0%> (-0.53%) ⬇️
cpp/src/arrow/io/hdfs-internal.cc 33.62% <0%> (-0.45%) ⬇️
... and 52 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6646864...84f4935. Read the comment docs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make notification_fd = -1 by default if not set (and get rid of the boost dependency)

Copy link
Contributor

@pcmoritz pcmoritz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me! Can you get rid of the boost::option dependency and use the value -1 instead of boost::none? At the moment plasma doesn't depend on boost and I'd like to keep it that way unless there is a good reason to introduce it.

Also can you rebase the PR? There is a small conflict at the moment.

@zhijunfu zhijunfu force-pushed the refactor-notification branch from 38d992b to 51b292e Compare May 16, 2018 15:26
@zhijunfu
Copy link
Contributor Author

@pcmoritz Thanks for reviewing :) Changed boost::none to -1 and did a rebase to resolve the conflict.

@pcmoritz
Copy link
Contributor

@zhijunfu Sorry, the other PR caused a merge conflict, can you rebase/merge so we can get this PR in? Thanks :)

@zhijunfu zhijunfu force-pushed the refactor-notification branch from 3199eea to f2377f8 Compare May 28, 2018 05:02
@zhijunfu
Copy link
Contributor Author

@pcmoritz Sorry for the long delay in reply, I just rebased the change on top of latest code. The build failures w/ parquet and tool-chain should be unrelated with the change.


/// The file descriptor used to push notifications to client. This is only valid
/// if client subscribes to plasma store. -1 indicates invalid.
int notification_fd;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated to the content of this patch: could we regularize the variable name convention in this header and elsewhere in Plasma? Would help with readability

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that makes a lot of sense:)

@zhijunfu
Copy link
Contributor Author

zhijunfu commented Jun 6, 2018

Re-triggered the build and it passed this time.

Copy link
Member

@wesm wesm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. Merging this, and also opened https://issues.apache.org/jira/browse/ARROW-2690

@wesm wesm closed this in 8156e25 Jun 9, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants