Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert some local scheduler data structures to C++ STL. #445

Merged
merged 6 commits into from
Apr 11, 2017
Merged

Convert some local scheduler data structures to C++ STL. #445

merged 6 commits into from
Apr 11, 2017

Conversation

robertnishihara
Copy link
Collaborator

@robertnishihara robertnishihara commented Apr 10, 2017

This makes some more progress on #404. In particular,

  • replaces some hash tables with std::unordered_map in the local scheduler
  • replaces some UT_arrays with std::vector in the local scheduler
  • improves the efficiency of some of the array operations by using the trick of removing an element from the middle by swapping it with the last element and popping the last element

@AmplabJenkins
Copy link

Merged build finished. Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/547/
Test PASSed.

@AmplabJenkins
Copy link

Merged build finished. Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/551/
Test PASSed.

@AmplabJenkins
Copy link

Merged build finished. Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/552/
Test PASSed.

@AmplabJenkins
Copy link

Merged build finished. Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/553/
Test PASSed.

"Tried to kill worker that doesn't exist");
auto it = std::find(state->workers.begin(), state->workers.end(), worker);
CHECK(it != state->workers.end());
std::swap(*it, state->workers.back());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only want to do this if it is not equal to the last element.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For efficiency reasons? The swap should be pretty cheap and this would add an extra if all of the time, right?


/* Make sure that we removed the worker. */
it = std::find(state->workers.begin(), state->workers.end(), worker);
CHECK(it == state->workers.end());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check doesn't really seem necessary anymore.

@@ -78,28 +77,25 @@ int force_kill_worker(event_loop *loop, timer_id id, void *context) {
* to clean up its own state.
* @return Void.
*/
void kill_worker(LocalSchedulerClient *worker, bool cleanup) {
void kill_worker(LocalSchedulerState *state,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to state as an argument here (LocalSchedulerClients should have a reference to it).

auto it = std::find(algorithm_state->executing_workers.begin(),
algorithm_state->executing_workers.end(), worker);
if (it != algorithm_state->executing_workers.end()) {
std::swap(*it, algorithm_state->executing_workers.back());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here for if the pointers are the same. It would be good to have a helper function for this.

/** A hash map of the objects that are not available locally. These are
* currently being fetched by this local scheduler. The key is the object
* ID. Every LOCAL_SCHEDULER_FETCH_TIMEOUT_MILLISECONDS, a Plasma fetch
* request will be sent the object IDs in this table. Each entry also holds
* an array of queued tasks that are dependent on it. */
object_entry *remote_objects;
std::unordered_map<ObjectID, ObjectEntry *, UniqueIDHasher> remote_objects;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one and local_objects can just be std::unordered_map<ObjectID, ObjectEntry, UniqueIDHasher>, to avoid allocation/deallocation.

@AmplabJenkins
Copy link

Merged build finished. Test FAILed.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/554/
Test FAILed.

@AmplabJenkins
Copy link

Merged build finished. Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/555/
Test PASSed.

@AmplabJenkins
Copy link

Merged build finished. Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/556/
Test PASSed.

@AmplabJenkins
Copy link

Merged build finished. Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/557/
Test PASSed.

@AmplabJenkins
Copy link

Merged build finished. Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/558/
Test PASSed.

@AmplabJenkins
Copy link

Merged build finished. Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/559/
Test PASSed.

@stephanie-wang stephanie-wang merged commit fb4525f into ray-project:master Apr 11, 2017
@stephanie-wang stephanie-wang deleted the morecpp branch April 11, 2017 04:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants