feat(agent)!: enable global agent pod for workflows Fixes #7891 #15243
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gaurang9991
force-pushed
the
global-agent-pod
branch
4 times, most recently
from
e73018c to
9de1a7d
Compare
Enable global agent pods that can execute tasks from multiple workflows using label selectors instead of requiring one agent pod per workflow and giveing users option to cutomize lifecyle behaviour of agent pods. BREAKING CHANGE: The ARGO_WORKFLOW_UID environment variable is no longer used. Workflow UIDs are now dynamically extracted from WorkflowTaskSet owner references at runtime. Signed-off-by: Gaurang Mishra <gaurangmishra48@gmail.com>
gaurang9991
force-pushed
the
global-agent-pod
branch
from
9de1a7d to
13ddc92
Compare
Joibel
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Enable global agent pods that can execute tasks from multiple workflows using label selectors instead of requiring one agent pod per workflow and giveing users option to cutomize lifecyle behaviour of agent pods.
BREAKING CHANGE: The ARGO_WORKFLOW_UID environment variable is no longer used. Workflow UIDs are now dynamically extracted from WorkflowTaskSet owner references at runtime.
Fixes #7891
Motivation
Motivation for the changes are to cutomize and utlize executor plugin to run high-frequency batch tasks without creating multiplepods.
Verification
Verification done in local environment and unit test
Senerios :
Default behaviour (current behaviour)- One agent pod per workflow
RunMultipleWorkflows: true – creates a agent pod with name agent-{service account name}. Now new pod is created when new workflow with same SA is submitted. The agent pod is deleted once all running workflows are completed.
Workflows with different service account create different agent pods
deleteAfterCompletion: false && RunMultipleWorkflows: true -
creates a agent pod with name agent-{service account name}. Now new pod is created when new workflow with same SA is submitted. The agent pod is not deleted after workflow completion. New agent pod is created when new plugins are added.
createPod: false – Controller now longer creates the agent pod and only reply on task set updates. Users can create their on logic or operators to execute and update taskset.
Documentation
created global-agent-pod.md file with doc and configuration details