Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

some questions about the schedulers. #2752

Open
wangqu123456 opened this issue Sep 22, 2023 · 3 comments
Open

some questions about the schedulers. #2752

wangqu123456 opened this issue Sep 22, 2023 · 3 comments
Assignees
Labels
help wanted Extra attention is needed

Comments

@wangqu123456
Copy link

first, if the schedulers in one scheduler cluster will work together? I mean that one could see task processed by another scheduler.
cause I review the code of task Manager, there are no share memory,maybe I overlook. waiting for anybody's answer。

@gaius-qi
Copy link
Member

@wangqu123456 Can you describe it in more detail?

@wangqu123456
Copy link
Author

I'm learning this project.
the task is the abstract of one time download in dragonfly; dfdaemon will register task to scheduler, scheduler will get info from task Manager to decide whether this task could be reused; My problem just in this place, in k8s deployment condition, schedulers running alone in each pod, and task Manager init behind the scheduler pod init; if one scheduler complete its task, then another scheduler also need to process this task, this task will be reused? how could two seperated scheduler could communicate their task infos?

@gaius-qi
Copy link
Member

image

@wangqu123456 Use task id to do consistent hashing to determine which scheduler to request.

@gaius-qi gaius-qi self-assigned this Sep 25, 2023
@gaius-qi gaius-qi added the help wanted Extra attention is needed label Sep 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants