Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove CoreWorkerTaskExecutionInterface #6009

Merged
merged 34 commits into from
Oct 25, 2019
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
649ad0f
Remove task_context
edoakes Oct 23, 2019
d656bae
Use core worker job ID
edoakes Oct 23, 2019
849b1e6
Fix put_object calls
edoakes Oct 23, 2019
c6fb56e
Remove compute_put_id
edoakes Oct 23, 2019
b7c034c
Postcrement
edoakes Oct 24, 2019
818a7a3
Temporary fix
edoakes Oct 24, 2019
34b45d2
Fix memory limits
edoakes Oct 25, 2019
7009cf8
Fix already exists check
edoakes Oct 25, 2019
15c457f
move actor_id into core worker
edoakes Oct 25, 2019
ed3d914
Remove setters from cython
edoakes Oct 25, 2019
e77aa6f
Add multithreading warning in C++
edoakes Oct 25, 2019
306c9de
Fix multithreading warning
edoakes Oct 25, 2019
e071cac
Fix core worker test
edoakes Oct 25, 2019
3d3f6c2
Reduce actor submission python overhead (#5949)
pcmoritz Oct 23, 2019
742e1dc
rllib] Fix leak of TensorFlow assign operations in DQN/DDPG
ericl Oct 23, 2019
f6a353e
Update release doc (#5988)
edoakes Oct 24, 2019
7b8021b
Ignore errors in ObjectID.__dealloc__ (#5997)
edoakes Oct 24, 2019
28e3cc5
Update profiling numbers (#5989)
edoakes Oct 25, 2019
005f7d4
fix comment (#6006)
edoakes Oct 25, 2019
2c94023
Use core_worker_ field
edoakes Oct 25, 2019
2138b17
Move even more into core worker
edoakes Oct 25, 2019
fda4ad3
Move ExecuteTask
edoakes Oct 25, 2019
39a5d78
Remove TaskExecutionInterface
edoakes Oct 25, 2019
7f8a9bb
Remove object interface unique_ptr
edoakes Oct 25, 2019
fa7c7d6
Merge branch 'master' into remove-execution
edoakes Oct 25, 2019
1dc6e9b
Remove gcs client unique_ptr
edoakes Oct 25, 2019
06dc2fb
Fix java
edoakes Oct 25, 2019
0c6ee87
Merge branch 'master' into remove-execution
edoakes Oct 25, 2019
f546982
Re-remove files
edoakes Oct 25, 2019
bb55a05
Fix mock worker
edoakes Oct 25, 2019
6a057ad
Remove CoreWorkerTaskSubmissionInterface
edoakes Oct 25, 2019
2dc49d8
Remove unused StopExecutingTasks
edoakes Oct 25, 2019
cd7ca2f
Remove StopExecutingTasks from cython
edoakes Oct 25, 2019
7d3fc0a
Organize fields and methods
edoakes Oct 25, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add multithreading warning in C++
  • Loading branch information
edoakes committed Oct 25, 2019
commit e77aa6f4a713cd7bb76408851ec7fe884c610aaa
12 changes: 9 additions & 3 deletions src/ray/core_worker/context.cc
Original file line number Diff line number Diff line change
Expand Up @@ -79,10 +79,8 @@ const TaskID &WorkerContext::GetCurrentTaskID() const {
return GetThreadContext().GetCurrentTaskID();
}

// TODO(edoakes): remove this once Python core worker uses the task interfaces.
void WorkerContext::SetCurrentJobId(const JobID &job_id) { current_job_id_ = job_id; }

// TODO(edoakes): remove this once Python core worker uses the task interfaces.
void WorkerContext::SetCurrentTaskId(const TaskID &task_id) {
GetThreadContext().SetCurrentTaskId(task_id);
}
Expand Down Expand Up @@ -123,9 +121,17 @@ bool WorkerContext::CurrentActorUseDirectCall() const {
return current_actor_use_direct_call_;
}

WorkerThreadContext &WorkerContext::GetThreadContext() {
WorkerThreadContext &WorkerContext::GetThreadContext(bool for_main_thread) {
if (thread_context_ == nullptr) {
thread_context_ = std::unique_ptr<WorkerThreadContext>(new WorkerThreadContext());
if (!for_main_thread && !multithreading_warning_printed_) {
std::cout << "WARNING: "
<< "Calling ray.get or ray.wait in a separate thread "
<< "may lead to deadlock if the main thread blocks on "
<< "this thread and there are not enough resources to "
<< "execute more tasks." << std::endl;
multithreading_warning_printed_ = true;
}
}

return *thread_context_;
Expand Down
6 changes: 5 additions & 1 deletion src/ray/core_worker/context.h
Original file line number Diff line number Diff line change
Expand Up @@ -47,8 +47,12 @@ class WorkerContext {
ActorID current_actor_id_;
bool current_actor_use_direct_call_;

// Flag used to ensure that we only print a warning about multithreading once per
// process.
static bool multithreading_warning_printed_;

private:
static WorkerThreadContext &GetThreadContext();
static WorkerThreadContext &GetThreadContext(bool for_main_thread = false);

/// Per-thread worker context.
static thread_local std::unique_ptr<WorkerThreadContext> thread_context_;
Expand Down