Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update tasks-and-workers.md #177

Merged
merged 1 commit into from
Nov 1, 2022
Merged

Update tasks-and-workers.md #177

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 11 additions & 20 deletions docs/getting-started/concepts/tasks-and-workers.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,30 +5,21 @@ sidebar_position: 1
# Workers and Tasks

## Worker
A worker is responsible for executing a task. Operator and System tasks are handled by the Conductor server. User defined tasks need to have a worker running in a different environment. This worker will poll Conductor's task queue to see if it has any work scheduled by the server.

Workers can be implemented in any language, and Conductor's [SDKs](/content/docs/how-tos/SDKs) provide support for worker framework that provides features such as polling threads, metrics and server communication that makes creating workers easy.
A worker is responsible for executing a task. Operators and System tasks are handled by the Conductor server, whereas the user-defined tasks need to have a worker running in a different environment. This worker will poll the Conductor's task queue to see if any work has been scheduled by the server. If there is a scheduled task, the worker executes the task and produces the output, provided input is specified. The worker’s output is sent back to the workflow and the process continues as per the defined workflow.

Each worker embodies Microservice design pattern and follows certain basic principles:
Workers can be implemented in any language. Conductor's [SDKs](https://orkes.io/content/docs/how-tos/SDKs) provide support for worker frameworks that deliver features such as polling threads, metrics and server communication which aids in the worker creation process.

1. Workers are stateless and do not implement a workflow specific logic.
2. Each worker executes a very specific task and produces well defined output given specific inputs.
3. Workers are meant to be idempotent (or should handle cases where the task that partially executed gets rescheduled due to timeouts etc.)
4. Workers do not implement the logic to handle retries etc, that is taken care by the Conductor server.
Each worker embodies the microservice design pattern and follows certain basic principles:

## Tasks
Tasks are the building blocks of workflow in Conductor. A task can be an [operator](operators), [system task](system-tasks)
or custom code [Worker](#worker) written in **any** programming language.

A typical Conductor workflow is a list of tasks that are executed until completion or the termination of the workflow.
* Workers are stateless and do not implement any workflow-specific logic.
* Each worker executes a particular task and produces well-defined output given specific inputs are provided.
* Workers are meant to be idempotent (Or should be able to handle cases where the partially executed tasks get rescheduled due to timeouts).
* Workers do not implement the logic to handle retries etc, which the Conductor server takes care of.

## Tasks

To learn more about tasks:
Tasks are the building blocks of a workflow in Conductor. A task can be an [operator](https://orkes.io/content/docs/getting-started/concepts/operators), [system task](https://orkes.io/content/docs/getting-started/concepts/system-tasks)
or custom code [Worker](https://orkes.io/content/docs/how-tos/Workers/write-workers) written in **any** programming language. A typical Conductor workflow is a list of tasks that are executed until completion or the termination of the workflow.

* [Creating tasks](/content/docs/how-tos/Tasks/creating-tasks)
* [Updating tasks](/content/docs/how-tos/Tasks/updating-tasks)
* [Reusing tasks](/content/docs/how-tos/Tasks/reusing-tasks)
* [Task to Domain](/content/docs/how-tos/Tasks/reusing-tasks)
* [Task Configurations](/content/docs/how-tos/Tasks/task-configurations)
* [Task Input Parameters](/content/docs/how-tos/Tasks/task-inputs)
* [Task Lifecycle](/content/docs/how-tos/Tasks/task-lifecycle)
Before being implemented in the workflow, you must register all tasks with the Conductor. You can also reuse a task within multiple workflows. You can begin with [creating tasks](https://orkes.io/content/docs/how-tos/Tasks/creating-tasks), and at a later point, you have the provision to [update](https://orkes.io/content/docs/how-tos/Tasks/updating-tasks)/[reuse](https://orkes.io/content/docs/how-tos/Tasks/reusing-tasks) them. You can also leverage the [task to domain](https://orkes.io/content/docs/how-tos/Tasks/task-domains) functionality where the task debugging and development can be done in the production environment. In addition, the transition between the different states of a task can be visualized from a [task’s lifecycle](https://orkes.io/content/docs/how-tos/Tasks/task-lifecycle).