Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Job orchestration 作业编排 #532

Closed
heziai opened this issue Nov 5, 2018 · 0 comments
Closed

Job orchestration 作业编排 #532

heziai opened this issue Nov 5, 2018 · 0 comments
Assignees
Milestone

Comments

@heziai
Copy link
Member

heziai commented Nov 5, 2018

背景需求
作业A执行完,触发作业B执行,触发作业C执行,以此类推。

实现方案
新增作业类型PASSIVE_JAVA_JOB和PASSIVE_SHELL_JOB,它们不具有cron表达式,完全依赖runAtOnce来触发执行。

作业新增upStream和downStream配置,分配对应配置上游和下游,多个作业使用英文逗号隔开。

为了以后升级、兼容性考虑,作业A执行后,调用saturn-console的rest api立即执行一次接口来触发作业B。这样,以后触发的逻辑的改动,修改rest api即可,无需executor的升级。

后台校验上下游配置,不能形成环。

触发下游时,可以携带一些信息,例如可以携带调用链信息,把整个调用链串起来。

不再使用依赖作业(dependencies),界面去掉该配置,去掉对启用/禁用作业时对依赖作业的判断。

界面展示域下作业的依赖图:作业总览和作业配置页面,提供一个按钮,点击按钮即弹出页面,展示依赖图。

作业配置界面展示作业类型,作业的新增、复制、导入、导出等功能对“被动作业”的支持。

上游执行成功才触发下游,不支持失败触发,因为executor网络问题导致的作业强杀、也有可能调不通saturn-console的api。

暂停作业时间段对被动作业无用,只对定时作业有用。前台隐藏被动作业的暂停时间段配置。

rerun对被动作业无用。前台隐藏该配置。

更新作业时,级联更新与该作业相关的上下游配置。

删除作业时,如果该作业配置了上下游,则报错、先断开该作业相关的上下游关系,才能进行删除。

上游作业选择框tips:

满足上游作业的条件:
1、只能是定时作业或被动作业
2、分片只能为1
3、不能是本地模式作业
4、不能是本作业的直接或者间接下游

下游作业选择框tips:

配置下游作业的条件:
1、只能是定时作业或被动作业
2、分片只能为1
3、不能是本地模式作业
满足下游作业的条件:
1、只能是被动作业
2、不能是本作业的直接或者间接上游

前台更新作业的本地模式、分片数、下游时,需要做校验。不仅仅在配置下游作业时做校验,也要在对本地模式、分片数配置时做校验,比如:已经配置下游作业,不能设置勾选本地模式、不能设置分片数大于1。

@heziai heziai added the feature label Nov 5, 2018
@heziai heziai added this to the 3.3.0 milestone Nov 5, 2018
@heziai heziai self-assigned this Nov 5, 2018
heziai added a commit that referenced this issue Nov 5, 2018
heziai added a commit that referenced this issue Nov 5, 2018
heziai added a commit that referenced this issue Nov 5, 2018
@heziai heziai changed the title job workflow function job arranging function Nov 7, 2018
heziai added a commit that referenced this issue Nov 8, 2018
…ository methods; job add/import/export support downStream; dependencies deprecated on GetJobConfig;
heziai added a commit that referenced this issue Nov 8, 2018
heziai added a commit that referenced this issue Nov 8, 2018
heziai added a commit that referenced this issue Nov 12, 2018
heziai added a commit that referenced this issue Nov 26, 2018
heziai added a commit that referenced this issue Nov 26, 2018
heziai added a commit that referenced this issue Nov 27, 2018
allen8203 added a commit that referenced this issue Nov 30, 2018
allen8203 added a commit that referenced this issue Nov 30, 2018
allen8203 added a commit that referenced this issue Dec 4, 2018
heziai added a commit that referenced this issue Dec 6, 2018
allen8203 added a commit that referenced this issue Dec 6, 2018
heziai added a commit that referenced this issue Dec 6, 2018
allen8203 added a commit that referenced this issue Dec 7, 2018
allen8203 added a commit that referenced this issue Dec 7, 2018
allen8203 added a commit that referenced this issue Dec 10, 2018
allen8203 added a commit that referenced this issue Dec 10, 2018
allen8203 added a commit that referenced this issue Dec 17, 2018
allen8203 added a commit that referenced this issue Dec 17, 2018
@kfchu kfchu changed the title job arranging function Job orchestration Dec 21, 2018
@kfchu kfchu changed the title Job orchestration Job orchestration 作业编排 Dec 21, 2018
heziai added a commit that referenced this issue Dec 21, 2018
@heziai heziai added done and removed done labels Dec 24, 2018
@heziai heziai added done and removed done labels Dec 24, 2018
heziai added a commit that referenced this issue Dec 24, 2018
@heziai heziai added the done label Dec 24, 2018
@kfchu kfchu closed this as completed Feb 14, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants