A monthly opportunity for users and maintainers of Workflows and Events to share their current work and
-hear about what’s coming on the roadmap. Please join us! For Community Meeting information, minutes and recordings
-please see here.
A weekly opportunity for committers and maintainers of Workflows and Events to discuss their current work and
-talk about what’s next. Feel free to join us! For Contributor Meeting information, minutes and recordings
-please see here.
The following diagram shows the components of the Argo Workflows architecture. There are two Deployments: Workflow Controller and Argo Server. The former does all of the reconciling, and the latter serves the API. Note that the Controller can be used stand alone.
The diagram below provides a little more detail as far as namespaces. The Workflow Controller and Argo Server both run in the argo namespace. Assuming Argo Workflows was installed as a Cluster Install or as a Managed Namespace Install (described here), the Workflows and the Pods generated from them run in a separate namespace.
-
The internals of a Pod are also shown. Each Step and each DAG Task cause a Pod to be generated, and each of these is composed of 3 containers:
-
-
main container runs the Image that the user indicated, where the argoexec utility is volume mounted and serves as the main command which calls the configured Command as a sub-process
-
init container is an InitContainer, fetching artifacts and parameters and making them available to the main container
-
wait container performs tasks that are needed for clean up, including saving off parameters and artifacts
The following diagram shows the process for reconciliation, whereby a set of worker goroutines process the Workflows which have been added to a Workflow queue based on adds and updates to Workflows and Workflow Pods. Note that in addition to the Informers shown, there are Informers for the other CRDs that Argo Workflows uses as well. You can find this code in workflow/controller/controller.go. Note that the controller only ever processes a single Workflow at a time.
-
-
-
Various configurations for Argo UI and Argo Server¶
-
The top diagram below shows what happens if you run "make start UI=true" locally (recommended if you need the UI during local development). This runs a React application (Webpack HTTP server) locally which serves the index.html and typescript files from port 8080. From the typescript code there are calls made to the back end API (Argo Server) at port 2746. The Webpack HTTP server is configured for hot reload, meaning the UI will update automatically based on local code changes.
-
The second diagram is an alternative approach for rare occasions that the React files are broken and you're doing local development. In this case, everything is served from the Argo Server at port 2746.
-
The third diagram shows how things are configured for a Kubernetes environment. It is similar to the second diagram in that the Argo Server hosts everything for the UI.
You can choose which kube config the Argo Server uses:
-
-
server - in hosted mode, use the kube config of service account, in local mode, use your local kube config.
-
client - requires clients to provide their Kubernetes bearer token and use that.
-
sso - since v2.9, use single sign-on, this will use the same service account as per "server" for RBAC. We expect to change this in the future so that the OAuth claims are mapped to service accounts.
-
-
The server used to start with auth mode of "server" by default, but since v3.0 it defaults to the "client".
-
To change the server auth mode specify the list as multiple auth-mode flags:
It is possible to have the Argo Workflows Server use the Argo CD Dex instance for authentication, for instance if you use Okta with SAML which cannot integrate with Argo Workflows directly. In order to make this happen, you will need the following:
-
-
You must be using at least Dex v2.35.0, because that's when staticClients[].secretEnv was added. That means Argo CD 1.7.12 and above.
-
A secret containing two keys, client-id and client-secret to be used by both Dex and Argo Workflows Server. client-id is argo-workflows-sso in this example, client-secret can be any random string. If Argo CD and Argo Workflows are installed in different namespaces the secret must be present in both of them. Example:
-
-
apiVersion:v1
-kind:Secret
-metadata:
-name:argo-workflows-sso
-data:
-# client-id is 'argo-workflows-sso'
-client-id:YXJnby13b3JrZmxvd3Mtc3Nv
-# client-secret is 'MY-SECRET-STRING-CAN-BE-UUID'
-client-secret:TVktU0VDUkVULVNUUklORy1DQU4tQkUtVVVJRA==
-
-
-
--auth-mode=sso server argument added
-
A Dex staticClients configured for argo-workflows-sso
-
The sso configuration filled out in Argo Workflows Server to match
-
-
Example manifests for authenticating against Argo CD's Dex (Kustomize)¶
-
In Argo CD, add an environment variable to Dex deployment and configuration:
-
---
-apiVersion:apps/v1
-kind:Deployment
-metadata:
-name:argocd-dex-server
-spec:
-template:
-spec:
-containers:
--name:dex
-env:
--name:ARGO_WORKFLOWS_SSO_CLIENT_SECRET
-valueFrom:
-secretKeyRef:
-name:argo-workflows-sso
-key:client-secret
----
-apiVersion:v1
-kind:ConfigMap
-metadata:
-name:argocd-cm
-data:
-# Kustomize sees the value of dex.config as a single string instead of yaml. It will not merge
-# Dex settings, but instead it will replace the entire configuration with the settings below,
-# so add these to the existing config instead of setting them in a separate file
-dex.config:|
-# Setting staticClients allows Argo Workflows to use Argo CD's Dex installation for authentication
-staticClients:
-- id: argo-workflows-sso
-name: Argo Workflow
-redirectURIs:
-- https://argo-workflows.mydomain.com/oauth2/callback
-secretEnv: ARGO_WORKFLOWS_SSO_CLIENT_SECRET
-
-
Note that the id field of staticClients must match the client-id.
-
In Argo Workflows add --auth-mode=sso argument to argo-server deployment.
-
---
-apiVersion:apps/v1
-kind:Deployment
-metadata:
-name:argo-server
-spec:
-template:
-spec:
-containers:
--name:argo-server
-args:
--server
----auth-mode=sso
----
-apiVersion:v1
-kind:ConfigMap
-metadata:
-name:workflow-controller-configmap
-data:
-# SSO Configuration for the Argo server.
-# You must also start argo server with `--auth-mode sso`.
-# https://argoproj.github.io/argo-workflows/argo-server-auth-mode/
-sso:|
-# This is the root URL of the OIDC provider (required).
-issuer: https://argo-cd.mydomain.com/api/dex
-# This is name of the secret and the key in it that contain OIDC client
-# ID issued to the application by the provider (required).
-clientId:
-name: argo-workflows-sso
-key: client-id
-# This is name of the secret and the key in it that contain OIDC client
-# secret issued to the application by the provider (required).
-clientSecret:
-name: argo-workflows-sso
-key: client-secret
-# This is the redirect URL supplied to the provider (required). It must
-# be in the form <argo-server-root-url>/oauth2/callback. It must be
-# browser-accessible.
-redirectUrl: https://argo-workflows.mydomain.com/oauth2/callback
-
-
Example Helm chart configuration for authenticating against Argo CD's Dex¶
server:
-extraArgs:
----auth-mode=sso
-sso:
-issuer:https://argo-cd.mydomain.com/api/dex
-# sessionExpiry defines how long your login is valid for in hours. (optional, default: 10h)
-sessionExpiry:240h
-clientId:
-name:argo-workflows-sso
-key:client-id
-clientSecret:
-name:argo-workflows-sso
-key:client-secret
-redirectUrl:https://argo-workflows.mydomain.com/oauth2/callback
-
It is possible to use Dex for authentication. This document describes how to set up Argo Workflows and Argo CD so that Argo Workflows uses Argo CD's Dex server for authentication.
Firstly, configure the settings workflow-controller-configmap.yaml with the correct OAuth 2 values. If working towards an OIDC configuration the Argo CD project has guides on its similar (though different) process for setting up OIDC providers. It also includes examples for specific providers. The main difference is that the Argo CD docs mention that their callback address endpoint is /auth/callback. For Argo Workflows, the default format is /oauth2/callback as shown in this comment in the default values.yaml file in the helm chart.
-
Next, create the Kubernetes secrets for holding the OAuth2 client-id and client-secret. You may refer to the kubernetes documentation on Managing secrets. For example by using kubectl with literals:
As of v2.12 we issue a JWE token for users rather than give them the ID token from your OAuth2 provider. This token is opaque and has a longer expiry time (10h by default).
-
The token encryption key is automatically generated by the Argo Server and stored in a Kubernetes secret name sso.
-
You can revoke all tokens by deleting the encryption key and restarting the Argo Server (so it generates a new key).
-
kubectldeletesecretsso
-
-
-
Warning
-
The old key will be in the memory the any running Argo Server, and they will therefore accept and user with token encrypted using the old key. Every Argo Server MUST be restarted.
You can optionally add RBAC to SSO. This allows you to give different users different access levels. Except for client auth mode, all users of the Argo Server must ultimately use a service account. So we allow you to define rules that map a user (maybe using their OIDC groups) to a service account in the same namespace as argo server by annotating the service account.
-
To allow service accounts to manage resources in other namespaces create a role and role binding in the target namespace.
-
RBAC config is installation-level, so any changes will need to be made by the team that installed Argo. Many complex rules will be burdensome on that team.
-
Firstly, enable the rbac: setting in workflow-controller-configmap.yaml. You likely want to configure RBAC using groups, so add scopes: to the SSO settings:
Not all OIDC providers support the groups scope. Please speak to your provider about their options.
-
-
To configure a service account to be used, annotate it:
-
apiVersion:v1
-kind:ServiceAccount
-metadata:
-name:admin-user
-annotations:
-# The rule is an expression used to determine if this service account
-# should be used.
-# * `groups` - an array of the OIDC groups
-# * `iss` - the issuer ("argo-server")
-# * `sub` - the subject (typically the username)
-# Must evaluate to a boolean.
-# If you want an account to be the default to use, this rule can be "true".
-# Details of the expression language are available in
-# https://github.com/antonmedv/expr/blob/master/docs/Language-Definition.md.
-workflows.argoproj.io/rbac-rule:"'admin'ingroups"
-# The precedence is used to determine which service account to use whe
-# Precedence is an integer. It may be negative. If omitted, it defaults to "0".
-# Numerically higher values have higher precedence (not lower, which maybe
-# counter-intuitive to you).
-# If two rules match and have the same precedence, then which one used will
-# be arbitrary.
-workflows.argoproj.io/rbac-rule-precedence:"1"
-
-
If no rule matches, we deny the user access.
-
Tip: You'll probably want to configure a default account to use if no other rule matches, e.g. a read-only account, you can do this as follows:
The precedence must be the lowest of all your service accounts.
-
As of Kubernetes v1.24, secrets for a service account token are no longer automatically created.
-Therefore, service account secrets for SSO RBAC must be created manually.
-See Manually create secrets for detailed instructions.
You can optionally configure RBAC SSO per namespace.
-Typically, on organization has a Kubernetes cluster and a central team (the owner of the cluster) manages the cluster. Along with this, there are multiple namespaces which are owned by individual teams. This feature would help namespace owners to define RBAC for their own namespace.
-
The feature is currently in beta.
-To enable the feature, set env variable SSO_DELEGATE_RBAC_TO_NAMESPACE=true in your argo-server deployment.
Configure a default account in the installation namespace that allows access to all users of your organization. This service account allows a user to login to the cluster. You could optionally add a workflow read-only role and role-binding.
All users MUST map to a cluster service account (such as the one above) before a namespace service account can apply.
-
-
Now, for the namespace that you own, configure a service account that allows members of your team to perform operations in your namespace.
-Make sure that the precedence of the namespace service account is higher than the precedence of the login service account. Create an appropriate role for this service account and bind it with a role-binding.
With this configuration, when a user is logged in via SSO, makes a request in my-namespace, and the rbac-rule matches, this service account allows the user to perform that operation. If no service account matches in the namespace, the first service account (user-default-login) and its associated role will be used to perform the operation.
By default, your SSO session will expire after 10 hours. You can change this by adding a sessionExpiry to your workflow-controller-configmap.yaml under the SSO heading.
-
sso:
-# Expiry defines how long your login is valid for in hours. (optional)
-sessionExpiry:240h
-
If your OIDC provider provides groups information with a claim name other than groups, you could configure config-map to specify custom claim name for groups. Argo now arbitrary custom claims and any claim can be used for expr eval. However, since group information is displayed in UI, it still needs to be an array of strings with group names as elements.
-
The customClaim in this case will be mapped to groups key and we can use the same key groups for evaluating our expressions
-
sso:
-# Specify custom claim name for OIDC groups.
-customGroupClaimName:argo_groups
-
-
If your OIDC provider provides groups information only using the user-info endpoint (e.g. Okta), you could configure userInfoPath to specify the user info endpoint that contains the groups claim.
You can configure filterGroupsRegex to filter the groups returned by the OIDC provider. Some use-cases for this include:
-
-
You have multiple applications using the same OIDC provider, and you only want to use groups that are relevant to Argo Workflows.
-
You have many groups and exceed the 4KB cookie size limit (cookies are used to store authentication tokens). If this occurs, login will fail.
-
-
sso:
-# Specify a list of regular expressions to filter the groups returned by the OIDC provider.
-# A logical "OR" is used between each regex in the list
-filterGroupsRegex:
--".*argo-wf.*"
--".*argo-workflow.*"
-
Since v3.0 the Argo Server listens for HTTPS requests, rather than HTTP.
-
-
The Argo Server is a server that exposes an API and UI for workflows. You'll need to use this if you want to offload large workflows or the workflow archive.
-
You can run this in either "hosted" or "local" mode.
If the server is running behind reverse proxy with a sub-path different from / (for example,
-/argo), you can set an alternative sub-path with the --basehref flag or the BASE_HREF
-environment variable.
Create a ingress, with the annotation ingress.kubernetes.io/rewrite-target: /:
-
-
If TLS is enabled (default in v3.0 and after), the ingress controller must be told
-that the backend uses HTTPS. The method depends on the ingress controller, e.g.
-Traefik expects an ingress.kubernetes.io/protocol annotation, while ingress-nginx
-uses nginx.ingress.kubernetes.io/backend-protocol
Argo Server does not perform authentication directly. It delegates this to either the Kubernetes API Server (when --auth-mode=client) and the OAuth provider (when --auth-mode=sso). In each case, it is recommended that the delegate implements any authentication rate limiting you need.
Argo Server does not log the IP addresses of API requests. We recommend you put the Argo Server behind a load balancer, and that load balancer is configured to log the IP addresses of requests that return authentication or authorization errors.
Argo Server by default rate limits to 1000 per IP per minute, you can configure it through --api-rate-limit. You can access additional information through the following headers.
-
-
X-Rate-Limit-Limit - the rate limit ceiling that is applicable for the current request.
-
X-Rate-Limit-Remaining - the number of requests left for the current rate-limit window.
-
X-Rate-Limit-Reset - the time at which the rate limit resets, specified in UTC time.
-
Retry-After - indicate when a client should retry requests (when the rate limit expires), in UTC time.
You can reduce duplication in your templates by configuring repositories that can be accessed by any workflow. This can also remove sensitive information from your templates.
-
Create a suitable config map in either (a) your workflows namespace or (b) in the managed namespace:
-
apiVersion:v1
-kind:ConfigMap
-metadata:
-# If you want to use this config map by default, name it "artifact-repositories". Otherwise, you can provide a reference to a
-# different config map in `artifactRepositoryRef.configMap`.
-name:my-artifact-repository
-annotations:
-# v3.0 and after - if you want to use a specific key, put that key into this annotation.
-workflows.argoproj.io/default-artifact-repository:default-v1-s3-artifact-repository
-data:
-default-v1-s3-artifact-repository:|
-s3:
-bucket: my-bucket
-endpoint: minio:9000
-insecure: true
-accessKeySecret:
-name: my-minio-cred
-key: accesskey
-secretKeySecret:
-name: my-minio-cred
-key: secretkey
-v2-s3-artifact-repository:|
-s3:
-...
-
-
You can override the artifact repository for a workflow as follows:
-
spec:
-artifactRepositoryRef:
-configMap:my-artifact-repository# default is "artifact-repositories"
-key:v2-s3-artifact-repository# default can be set by the `workflows.argoproj.io/default-artifact-repository` annotation in config map.
-
Files maybe shown in the UI. To determine if a file can be shown, the UI checks if the artifact's file extension is
-supported. The extension is found in the artifact's key.
We assume that artifacts are not trusted, so by default, artifacts are served with a Content-Security-Policy that
-disables JavaScript and remote files.
-
This is similar to what happens when you include third-party scripts, such as analytic tracking, in your website.
-However, those tracking codes are normally served from a different domain to your main website. Artifacts are served
-from the same origin, so normal browser controls are not secure enough.
Previously, users could access the artifacts of any workflows they could access. To allow HTML files to link to other files
-within their tree, you can now access any sub-paths of the artifact's key.
-
Example:
-
The artifact produces a folder in an S3 bucket named my-bucket, with a key report/. You can also access anything
-matching report/*.
The pattern involves two steps - the first step is a short-running step that triggers a long-running job outside Argo (e.g. an HTTP submission), and the second step is a suspend step that suspends workflow execution and is ultimately either resumed or stopped (i.e. failed) via a call to the Argo API when the job outside Argo succeeds or fails.
-
When implemented as a WorkflowTemplate it can look something like this:
In this case the job-cmd parameter can be a command that makes an HTTP call via curl to an endpoint that returns a job UUID. More sophisticated submission and parsing of submission output could be done with something like a Python script step.
-
On job completion the external job would need to call either resume if successful:
Requests are sent to the Argo Server API via GRPC (using HTTP/2). Large workflows and the workflow archive are supported. Network load-balancers that do not support HTTP/2 are not supported.
-
Use if you do not have access to the Kubernetes API (e.g. you're in another cluster), and you're running the Argo Server using a network load-balancer that support HTTP/2.
If you're have transport-layer security (TLS) enabled (i.e. you are running "argo server --secure" and therefore has HTTPS):
-
ARGO_SECURE=true
-
-
-
If your server is running with self-signed certificates. Do not use in production:
-
ARGO_INSECURE_SKIP_VERIFY=true
-
-
-
By default, the CLI uses your KUBECONFIG to determine default for ARGO_TOKEN and ARGO_NAMESPACE. You probably error with "no configuration has been provided". To prevent it:
-
KUBECONFIG=/dev/null
-
-
-
You will then need to set:
-
ARGO_NAMESPACE=argo
-
-
-
And:
-
ARGO_TOKEN='Bearer ******' ;# Should always start with "Bearer " or "Basic ".
-
As per GRPC mode, but uses HTTP. Can be used with ALB that does not support HTTP/2. The command "argo logs --since-time=2020...." will not work (due to time-type).
-
Use this when your network load-balancer does not support HTTP/2.
-
Use the same configuration as GRPC mode, but also set:
-
ARGO_HTTP1=true
-
-
-
If your server is behind an ingress with a path (you'll be running "argo server --basehref /...) or "BASE_HREF=/... argo server"):
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- -h, --help help for argo
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--chunk-size int Return large lists in chunks rather than all at once. Pass 0 to disable.
- -h, --help help for list
- -o, --output string Output format. One of: json|yaml|wide (default "wide")
- -l, --selector string Selector (label query) to filter on, not including uninitialized ones, supports '=', '==', and '!='.(e.g. -l key1=value1,key2=value2)
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--field-selector string Selector (field query) to filter on, supports '=', '==', and '!='.(e.g. --field-selector key1=value1,key2=value2). The server only supports a limited number of field queries per type.
- -h, --help help for resubmit
- --log log the workflow until it completes
- --memoized re-use successful steps & outputs from the previous run
- -o, --output string Output format. One of: name|json|yaml|wide
- -p, --parameter stringArray input parameter to override on the original workflow spec
- --priority int32 workflow priority
- -l, --selector string Selector (label query) to filter on, not including uninitialized ones, supports '=', '==', and '!='.(e.g. -l key1=value1,key2=value2)
- -w, --wait wait for the workflow to complete, only works when a single workflow is resubmitted
- --watch watch the workflow until it completes, only works when a single workflow is resubmitted
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--field-selector string Selector (field query) to filter on, supports '=', '==', and '!='.(e.g. --field-selector key1=value1,key2=value2). The server only supports a limited number of field queries per type.
- -h, --help help for retry
- --log log the workflow until it completes
- --node-field-selector string selector of nodes to reset, eg: --node-field-selector inputs.paramaters.myparam.value=abc
- -o, --output string Output format. One of: name|json|yaml|wide
- -p, --parameter stringArray input parameter to override on the original workflow spec
- --restart-successful indicates to restart successful nodes matching the --node-field-selector
- -l, --selector string Selector (label query) to filter on, not including uninitialized ones, supports '=', '==', and '!='.(e.g. -l key1=value1,key2=value2)
- -w, --wait wait for the workflow to complete, only works when a single workflow is retried
- --watch watch the workflow until it completes, only works when a single workflow is retried
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
# List Cluster Workflow Templates:
- argo cluster-template list
-
-# List Cluster Workflow Templates with additional details such as labels, annotations, and status:
- argo cluster-template list --output wide
-
-# List Cluster Workflow Templates by name only:
- argo cluster-template list -o name
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
Write bash or zsh shell completion code to standard output.
-
For bash, ensure you have bash completions installed and enabled.
-To access completions in your current shell, run
-$ source <(argo completion bash)
-Alternatively, write it to a file and source in .bash_profile
-
For zsh, output to a file in a directory referenced by the $fpath shell
-variable.
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
# Copy a workflow's artifacts to a local output directory:
-
- argo cp my-wf output-directory
-
-# Copy artifacts from a specific node in a workflow to a local output directory:
-
- argo cp my-wf output-directory --node-id=my-wf-node-id-123
-
--artifact-name string name of output artifact in workflow
- -h, --help help for cp
- -n, --namespace string namespace of workflow
- --node-id string id of node in workflow
- --path string use variables {workflowName}, {nodeId}, {templateName}, {artifactName}, and {namespace} to create a customized path to store the artifacts; example: {workflowName}/{templateName}/{artifactName} (default "{namespace}/{workflowName}/{nodeId}/outputs/{artifactName}")
- --template-name string name of template in workflow
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
-A, --all-namespaces Show workflows from all namespaces
- -h, --help help for list
- -o, --output string Output format. One of: wide|name
- -l, --selector string Selector (label query) to filter on, supports '=', '==', and '!='.(e.g. -l key1=value1,key2=value2). Matching objects must satisfy all of the specified label constraints.
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--all Delete all workflows
- -A, --all-namespaces Delete workflows from all namespaces
- --completed Delete completed workflows
- --dry-run Do not delete the workflow, only print what would happen
- --field-selector string Selector (field query) to filter on, supports '=', '==', and '!='.(e.g. --field-selector key1=value1,key2=value2). The server only supports a limited number of field queries per type.
- --force Force delete workflows by removing finalizers
- -h, --help help for delete
- --older string Delete completed workflows finished before the specified duration (e.g. 10m, 3h, 1d)
- --prefix string Delete workflows by prefix
- --query-chunk-size int Run the list query in chunks (deletes will still be executed individually)
- --resubmitted Delete resubmitted workflows
- -l, --selector string Selector (label query) to filter on, not including uninitialized ones, supports '=', '==', and '!='.(e.g. -l key1=value1,key2=value2)
- --status strings Delete by status (comma separated)
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
# Lint all manifests in a specified directory:
-
- argo lint ./manifests
-
-# Lint only manifests of Workflows and CronWorkflows from stdin:
-
- cat manifests.yaml | argo lint --kinds=workflows,cronworkflows -
-
-h, --help help for lint
- --kinds strings Which kinds will be linted. Can be: workflows|workflowtemplates|cronworkflows|clusterworkflowtemplates (default [all])
- --offline perform offline linting. For resources referencing other resources, the references will be resolved from the provided args
- -o, --output string Linting results output format. One of: pretty|simple (default "pretty")
- --strict Perform strict workflow validation (default true)
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
# List all workflows:
- argo list
-
-# List all workflows from all namespaces:
- argo list -A
-
-# List all running workflows:
- argo list --running
-
-# List all completed workflows:
- argo list --completed
-
- # List workflows created within the last 10m:
- argo list --since 10m
-
-# List workflows that finished more than 2h ago:
- argo list --older 2h
-
-# List workflows with more information (such as parameters):
- argo list -o wide
-
-# List workflows in YAML format:
- argo list -o yaml
-
-# List workflows that have both labels:
- argo list -l label1=value1,label2=value2
-
-A, --all-namespaces Show workflows from all namespaces
- --chunk-size int Return large lists in chunks rather than all at once. Pass 0 to disable.
- --completed Show completed workflows. Mutually exclusive with --running.
- --field-selector string Selector (field query) to filter on, supports '=', '==', and '!='.(e.g. --field-selector key1=value1,key2=value2). The server only supports a limited number of field queries per type.
- -h, --help help for list
- --no-headers Don't print headers (default print headers).
- --older string List completed workflows finished before the specified duration (e.g. 10m, 3h, 1d)
- -o, --output string Output format. One of: name|wide|yaml|json
- --prefix string Filter workflows by prefix
- --resubmitted Show resubmitted workflows
- --running Show running workflows. Mutually exclusive with --completed.
- -l, --selector string Selector (label query) to filter on, not including uninitialized ones, supports '=', '==', and '!='.(e.g. -l key1=value1,key2=value2)
- --since string Show only workflows created after than a relative duration
- --status strings Filter by status (comma separated)
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
# Print the logs of a workflow:
-
- argo logs my-wf
-
-# Follow the logs of a workflows:
-
- argo logs my-wf --follow
-
-# Print the logs of a workflows with a selector:
-
- argo logs my-wf -l app=sth
-
-# Print the logs of single container in a pod
-
- argo logs my-wf my-pod -c my-container
-
-# Print the logs of a workflow's pods:
-
- argo logs my-wf my-pod
-
-# Print the logs of a pods:
-
- argo logs --since=1h my-pod
-
-# Print the logs of the latest workflow:
- argo logs @latest
-
-c, --container string Print the logs of this container (default "main")
- -f, --follow Specify if the logs should be streamed.
- --grep string grep for lines
- -h, --help help for logs
- --no-color Disable colorized output
- -p, --previous Specify if the previously terminated container logs should be returned.
- -l, --selector string log selector for some pod
- --since duration Only return logs newer than a relative duration like 5s, 2m, or 3h. Defaults to all logs. Only one of since-time / since may be used.
- --since-time string Only return logs after a specific date (RFC3339). Defaults to all logs. Only one of since-time / since may be used.
- --tail int If set, the number of lines from the end of the logs to show. If not specified, logs are shown from the creation of the container or sinceSeconds or sinceTime (default -1)
- --timestamps Include timestamps on each line in the log output
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
# Set outputs to a node within a workflow:
-
- argo node set my-wf --output-parameter parameter-name="Hello, world!" --node-field-selector displayName=approve
-
-# Set the message of a node within a workflow:
-
- argo node set my-wf --message "We did it!"" --node-field-selector displayName=approve
-
-h, --help help for node
- -m, --message string Set the message of a node, eg: --message "Hello, world!"
- --node-field-selector string Selector of node to set, eg: --node-field-selector inputs.paramaters.myparam.value=abc
- -p, --output-parameter stringArray Set a "supplied" output parameter of node, eg: --output-parameter parameter-name="Hello, world!"
- --phase string Phase to set the node to, eg: --phase Succeeded
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--field-selector string Selector (field query) to filter on, supports '=', '==', and '!='.(e.g. --field-selector key1=value1,key2=value2). The server only supports a limited number of field queries per type.
- -h, --help help for resubmit
- --log log the workflow until it completes
- --memoized re-use successful steps & outputs from the previous run
- -o, --output string Output format. One of: name|json|yaml|wide
- -p, --parameter stringArray input parameter to override on the original workflow spec
- --priority int32 workflow priority
- -l, --selector string Selector (label query) to filter on, not including uninitialized ones, supports '=', '==', and '!='.(e.g. -l key1=value1,key2=value2)
- -w, --wait wait for the workflow to complete, only works when a single workflow is resubmitted
- --watch watch the workflow until it completes, only works when a single workflow is resubmitted
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--field-selector string Selector (field query) to filter on, supports '=', '==', and '!='.(e.g. --field-selector key1=value1,key2=value2). The server only supports a limited number of field queries per type.
- -h, --help help for retry
- --log log the workflow until it completes
- --node-field-selector string selector of nodes to reset, eg: --node-field-selector inputs.paramaters.myparam.value=abc
- -o, --output string Output format. One of: name|json|yaml|wide
- -p, --parameter stringArray input parameter to override on the original workflow spec
- --restart-successful indicates to restart successful nodes matching the --node-field-selector
- -l, --selector string Selector (label query) to filter on, not including uninitialized ones, supports '=', '==', and '!='.(e.g. -l key1=value1,key2=value2)
- -w, --wait wait for the workflow to complete, only works when a single workflow is retried
- --watch watch the workflow until it completes, only works when a single workflow is retried
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--access-control-allow-origin string Set Access-Control-Allow-Origin header in HTTP responses.
- --allowed-link-protocol stringArray Allowed link protocol in configMap. Used if the allowed configMap links protocol are different from http,https. Defaults to the environment variable ALLOWED_LINK_PROTOCOL (default [http,https])
- --api-rate-limit uint Set limit per IP for api ratelimiter (default 1000)
- --auth-mode stringArray API server authentication mode. Any 1 or more length permutation of: client,server,sso (default [client])
- --basehref string Value for base href in index.html. Used if the server is running behind reverse proxy under subpath different from /. Defaults to the environment variable BASE_HREF. (default "/")
- -b, --browser enable automatic launching of the browser [local mode]
- --configmap string Name of K8s configmap to retrieve workflow controller configuration (default "workflow-controller-configmap")
- --event-async-dispatch dispatch event async
- --event-operation-queue-size int how many events operations that can be queued at once (default 16)
- --event-worker-count int how many event workers to run (default 4)
- -h, --help help for server
- --hsts Whether or not we should add a HTTP Secure Transport Security header. This only has effect if secure is enabled. (default true)
- --kube-api-burst int Burst to use while talking with kube-apiserver. (default 30)
- --kube-api-qps float32 QPS to use while talking with kube-apiserver. (default 20)
- --log-format string The formatter to use for logs. One of: text|json (default "text")
- --managed-namespace string namespace that watches, default to the installation namespace
- --namespaced run as namespaced mode
- -p, --port int Port to listen on (default 2746)
- -e, --secure Whether or not we should listen on TLS. (default true)
- --tls-certificate-secret-name string The name of a Kubernetes secret that contains the server certificates
- --x-frame-options string Set X-Frame-Options header in HTTP responses. (default "DENY")
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--dry-run If true, only print the workflows that would be stopped, without stopping them.
- --field-selector string Selector (field query) to filter on, supports '=', '==', and '!='.(e.g. --field-selector key1=value1,key2=value2). The server only supports a limited number of field queries per type.
- -h, --help help for stop
- --message string Message to add to previously running nodes
- --node-field-selector string selector of node to stop, eg: --node-field-selector inputs.paramaters.myparam.value=abc
- -l, --selector string Selector (label query) to filter on, not including uninitialized ones, supports '=', '==', and '!='.(e.g. -l key1=value1,key2=value2)
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--dry-run modify the workflow on the client-side without creating it
- --entrypoint string override entrypoint
- --from kind/name Submit from an existing kind/name E.g., --from=cronwf/hello-world-cwf
- --generate-name string override metadata.generateName
- -h, --help help for submit
- -l, --labels string Comma separated labels to apply to the workflow. Will override previous values.
- --log log the workflow until it completes
- --name string override metadata.name
- --node-field-selector string selector of node to display, eg: --node-field-selector phase=abc
- -o, --output string Output format. One of: name|json|yaml|wide
- -p, --parameter stringArray pass an input parameter
- -f, --parameter-file string pass a file containing all input parameters
- --priority int32 workflow priority
- --scheduled-time string Override the workflow's scheduledTime parameter (useful for backfilling). The time must be RFC3339
- --server-dry-run send request to server with dry-run flag which will modify the workflow without creating it
- --serviceaccount string run all pods in the workflow using specified serviceaccount
- --status string Filter by status (Pending, Running, Succeeded, Skipped, Failed, Error). Should only be used with --watch.
- --strict perform strict workflow validation (default true)
- -w, --wait wait for the workflow to complete
- --watch watch the workflow until it completes
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--dry-run Do not terminate the workflow, only print what would happen
- --field-selector string Selector (field query) to filter on, supports '=', '==', and '!='.(e.g. --field-selector key1=value1,key2=value2). The server only supports a limited number of field queries per type.
- -h, --help help for terminate
- -l, --selector string Selector (label query) to filter on, not including uninitialized ones, supports '=', '==', and '!='.(e.g. -l key1=value1,key2=value2)
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
-h, --help help for watch
- --node-field-selector string selector of node to display, eg: --node-field-selector phase=abc
- --status string Filter by status (Pending, Running, Succeeded, Skipped, Failed, Error)
-
--argo-base-href string An path to use with HTTP client (e.g. due to BASE_HREF). Defaults to the ARGO_BASE_HREF environment variable.
- --argo-http1 If true, use the HTTP client. Defaults to the ARGO_HTTP1 environment variable.
- -s, --argo-server host:port API server host:port. e.g. localhost:2746. Defaults to the ARGO_SERVER environment variable.
- --as string Username to impersonate for the operation
- --as-group stringArray Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
- --as-uid string UID to impersonate for the operation
- --certificate-authority string Path to a cert file for the certificate authority
- --client-certificate string Path to a client certificate file for TLS
- --client-key string Path to a client key file for TLS
- --cluster string The name of the kubeconfig cluster to use
- --context string The name of the kubeconfig context to use
- --gloglevel int Set the glog logging level
- -H, --header strings Sets additional header to all requests made by Argo CLI. (Can be repeated multiple times to add multiple headers, also supports comma separated headers) Used only when either ARGO_HTTP1 or --argo-http1 is set to true.
- --insecure-skip-tls-verify If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
- -k, --insecure-skip-verify If true, the Argo Server's certificate will not be checked for validity. This will make your HTTPS connections insecure. Defaults to the ARGO_INSECURE_SKIP_VERIFY environment variable.
- --instanceid string submit with a specific controller's instance id label. Default to the ARGO_INSTANCEID environment variable.
- --kubeconfig string Path to a kube config. Only required if out-of-cluster
- --loglevel string Set the logging level. One of: debug|info|warn|error (default "info")
- -n, --namespace string If present, the namespace scope for this CLI request
- --password string Password for basic authentication to the API server
- --proxy-url string If provided, this URL will be used to connect via proxy
- --request-timeout string The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
- -e, --secure Whether or not the server is using TLS with the Argo Server. Defaults to the ARGO_SECURE environment variable. (default true)
- --server string The address and port of the Kubernetes API server
- --tls-server-name string If provided, this name will be used to validate server certificate. If this is not provided, hostname used to contact the server is used.
- --token string Bearer token for authentication to the API server
- --user string The name of the kubeconfig user to use
- --username string Username for basic authentication to the API server
- -v, --verbose Enabled verbose logging, i.e. --loglevel debug
-
This page contains an overview of the client libraries for using the Argo API from various programming languages.
-
To write applications using the REST API, you do not need to implement the API calls and request/response types
-yourself. You can use a client library for the programming language you are using.
-
Client libraries often handle common tasks such as authentication for you.
ClusterWorkflowTemplates are cluster scoped WorkflowTemplates. ClusterWorkflowTemplate
-can be created cluster scoped like ClusterRole and can be accessed across all namespaces in the cluster.
You can reference templates from other ClusterWorkflowTemplates using a templateRef field with clusterScope: true .
-Just as how you reference other templates within the same Workflow, you should do so from a steps or dag template.
-
Here is an example:
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:workflow-template-hello-world-
-spec:
-entrypoint:whalesay
-templates:
--name:whalesay
-steps:# You should only reference external "templates" in a "steps" or "dag" "template".
---name:call-whalesay-template
-templateRef:# You can reference a "template" from another "WorkflowTemplate or ClusterWorkflowTemplate" using this field
-name:cluster-workflow-template-whalesay-template# This is the name of the "WorkflowTemplate or ClusterWorkflowTemplate" CRD that contains the "template" you want
-template:whalesay-template# This is the name of the "template" you want to reference
-clusterScope:true# This field indicates this templateRef is pointing ClusterWorkflowTemplate
-arguments:# You can pass in arguments as normal
-parameters:
--name:message
-value:"helloworld"
-
-
-
2.9 and after
-
-
Create Workflow from ClusterWorkflowTemplate Spec¶
-
You can create Workflow from ClusterWorkflowTemplate spec using workflowTemplateRef with clusterScope: true. If you pass the arguments to created Workflow, it will be merged with cluster workflow template arguments
-
Here is an example for ClusterWorkflowTemplate with entrypoint and arguments
You can set Step/DAG level artifacts or parameters based on an expression.
-Use fromExpression under a Step/DAG level output artifact and expression under a Step/DAG level output parameter.
⚠️ We do not recommend you rely on Argo Workflows to archive logs. Instead, use a conventional Kubernetes logging facility.
-
To enable automatic pipeline logging, you need to configure archiveLogs at workflow-controller config-map, workflow spec, or template level. You also need to configure Artifact Repository to define where this logging artifact is stored.
To run Argo workflows that use artifacts, you must configure and use an artifact
-repository. Argo supports any S3 compatible artifact repository such as AWS, GCS
-and MinIO. This section shows how to configure the artifact repository.
-Subsequent sections will show how to use it.
-
-
-
-
Name
-
Inputs
-
Outputs
-
Garbage Collection
-
Usage (Feb 2020)
-
-
-
-
-
Artifactory
-
Yes
-
Yes
-
No
-
11%
-
-
-
Azure Blob
-
Yes
-
Yes
-
Yes
-
-
-
-
-
GCS
-
Yes
-
Yes
-
Yes
-
-
-
-
-
Git
-
Yes
-
No
-
No
-
-
-
-
-
HDFS
-
Yes
-
Yes
-
No
-
3%
-
-
-
HTTP
-
Yes
-
Yes
-
No
-
2%
-
-
-
OSS
-
Yes
-
Yes
-
No
-
-
-
-
-
Raw
-
Yes
-
No
-
No
-
5%
-
-
-
S3
-
Yes
-
Yes
-
Yes
-
86%
-
-
-
-
The actual repository used by a workflow is chosen by the following rules:
-
-
Anything explicitly configured using Artifact Repository Ref. This is the most flexible, safe, and secure option.
-
From a config map named artifact-repositories if it has the workflows.argoproj.io/default-artifact-repository annotation in the workflow's namespace.
First, install helm. Then, install MinIO with the below commands:
-
helmrepoaddminiohttps://helm.min.io/# official minio Helm charts
-helmrepoupdate
-helminstallargo-artifactsminio/minio--setservice.type=LoadBalancer--setfullnameOverride=argo-artifacts
-
-
Login to the MinIO UI using a web browser (port 9000) after obtaining the
-external IP using kubectl.
-
kubectlgetserviceargo-artifacts
-
-
On Minikube:
-
minikubeservice--urlargo-artifacts
-
-
NOTE: When MinIO is installed via Helm, it generates
-credentials, which you will use to login to the UI:
-Use the commands shown below to see the credentials
-
-
AccessKey: kubectl get secret argo-artifacts -o jsonpath='{.data.accesskey}' | base64 --decode
-
SecretKey: kubectl get secret argo-artifacts -o jsonpath='{.data.secretkey}' | base64 --decode
-
-
Create a bucket named my-bucket from the MinIO UI.
-
If MinIO is configured to use TLS you need to set the parameter insecure to false. Additionally, if MinIO is protected by certificates generated by a custom CA, you first need to save the CA certificate in a Kubernetes secret, then set the caSecret parameter accordingly. This will allow Argo to correctly verify the server certificate presented by MinIO. For example:
Create your bucket and access keys for the bucket. AWS access keys have the same
-permissions as the user they are associated with. In particular, you cannot
-create access keys with reduced scope. If you want to limit the permissions for
-an access key, you will need to create a user with just the permissions you want
-to associate with the access key. Otherwise, you can just create an access key
-using your existing user account.
If you do not have Artifact Garbage Collection configured, you should remove s3:DeleteObject from the list of Actions above.
-
NOTE: if you want argo to figure out which region your buckets belong in, you
-must additionally set the following statement policy. Otherwise, you must
-specify a bucket region in your workflow configuration.
If you wish to use S3 IRSA instead of passing in an accessKey and secretKey, you need to annotate the service account of both the running workflow (in order to save logs/artifacts) and the argo-server pod (in order to retrieve the logs/artifacts).
Create and download a Google Cloud service account key.
-
Create a kubernetes secret to store the key.
-
Configure gcs artifact as following in the yaml.
-
-
artifacts:
--name:message
-path:/tmp/message
-gcs:
-bucket:my-bucket-name
-key:path/in/bucket
-# serviceAccountKeySecret is a secret selector.
-# It references the k8s secret named 'my-gcs-credentials'.
-# This secret is expected to have have the key 'serviceAccountKey',
-# containing the base64 encoded credentials
-# to the bucket.
-#
-# If it's running on GKE and Workload Identity is used,
-# serviceAccountKeySecret is not needed.
-serviceAccountKeySecret:
-name:my-gcs-credentials
-key:serviceAccountKey
-
-
If it's a GKE cluster, and Workload Identity is configured, there's no need to
-create the service account key and store it as a Kubernetes secret,
-serviceAccountKeySecret is also not needed in this case. Please follow the
-link to configure Workload Identity
-(https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity).
artifacts:
--name:my-output-artifact
-path:/my-output-artifact
-s3:
-endpoint:storage.googleapis.com
-bucket:my-gcs-bucket-name
-# NOTE that, by default, all output artifacts are automatically tarred and
-# gzipped before saving. So as a best practice, .tgz or .tar.gz
-# should be incorporated into the key name so the resulting file
-# has an accurate file extension.
-key:path/in/bucket/my-output-artifact.tgz
-accessKeySecret:
-name:my-gcs-s3-credentials
-key:accessKey
-secretKeySecret:
-name:my-gcs-s3-credentials
-key:secretKey
-
Create your bucket and access key for the bucket. Suggest to limit the permission
-for the access key, you will need to create a user with just the permissions you
-want to associate with the access key. Otherwise, you can just create an access key
-using your existing user account.
-
Setup Alibaba Cloud CLI
-and follow the steps to configure the artifact storage for your workflow:
-
$exportmybucket=bucket-workflow-artifect
-$exportmyregion=cn-zhangjiakou
-$# limit permission to read/write the bucket.
-$cat>policy.json<<EOF
-{
- "Version": "1",
- "Statement": [
- {
- "Effect": "Allow",
- "Action": [
- "oss:PutObject",
- "oss:GetObject"
- ],
- "Resource": "acs:oss:*:*:$mybucket/*"
- }
- ]
-}
-EOF
-$# create bucket.
-$aliyunossmboss://$mybucket--region$myregion
-$# show endpoint of bucket.
-$aliyunossstatoss://$mybucket
-$#create a ram user to access bucket.
-$aliyunramCreateUser--UserName$mybucket-user
-$# create ram policy with the limit permission.
-$aliyunramCreatePolicy--PolicyName$mybucket-policy--PolicyDocument"$(catpolicy.json)"
-$# attch ram policy to the ram user.
-$aliyunramAttachPolicyToUser--UserName$mybucket-user--PolicyName$mybucket-policy--PolicyTypeCustom
-$# create access key and secret key for the ram user.
-$aliyunramCreateAccessKey--UserName$mybucket-user>access-key.json
-$# create secret in demo namespace, replace demo with your namespace.
-$kubectlcreatesecretgeneric$mybucket-credentials-ndemo\
---from-literal"accessKey=$(cataccess-key.json|jq-r.AccessKey.AccessKeyId)"\
---from-literal"secretKey=$(cataccess-key.json|jq-r.AccessKey.AccessKeySecret)"
-$# create configmap to config default artifact for a namespace.
-$cat>default-artifact-repository.yaml<< EOF
-apiVersion: v1
-kind: ConfigMap
-metadata:
- # If you want to use this config map by default, name it "artifact-repositories". Otherwise, you can provide a reference to a
- # different config map in `artifactRepositoryRef.configMap`.
- name: artifact-repositories
- annotations:
- # v3.0 and after - if you want to use a specific key, put that key into this annotation.
- workflows.argoproj.io/default-artifact-repository: default-oss-artifact-repository
-data:
- default-oss-artifact-repository: |
- oss:
- endpoint: http://oss-cn-zhangjiakou-internal.aliyuncs.com
- bucket: $mybucket
- # accessKeySecret and secretKeySecret are secret selectors.
- # It references the k8s secret named 'bucket-workflow-artifect-credentials'.
- # This secret is expected to have have the keys 'accessKey'
- # and 'secretKey', containing the base64 encoded credentials
- # to the bucket.
- accessKeySecret:
- name: $mybucket-credentials
- key: accessKey
- secretKeySecret:
- name: $mybucket-credentials
- key: secretKey
-EOF
-# create cm in demo namespace, replace demo with your namespace.
-$kapply-fdefault-artifact-repository.yaml-ndemo
-
-
You can also set createBucketIfNotPresent to true to tell the artifact driver to automatically create the OSS bucket if it doesn't exist yet when saving artifacts. Note that you'll need to set additional permission for your OSS account to create new buckets.
If you wish to use OSS RRSA instead of passing in an accessKey and secretKey, you need to perform the following actions:
-
-
Install pod-identity-webhook in your cluster to automatically inject the OIDC tokens and environment variables.
-
Add the label pod-identity.alibabacloud.com/injection: 'on' to the target workflow namespace.
-
Add the annotation pod-identity.alibabacloud.com/role-name: $your_ram_role_name to the service account of running workflow.
-
Set useSDKCreds: true in your target artifact repository cm and remove the secret references to AK/SK.
-
-
apiVersion:v1
-kind:Namespace
-metadata:
-name:my-ns
-labels:
-pod-identity.alibabacloud.com/injection:'on'
-
----
-apiVersion:v1
-kind:ServiceAccount
-metadata:
-name:my-sa
-namespace:rrsa-demo
-annotations:
-pod-identity.alibabacloud.com/role-name:$your_ram_role_name
-
----
-apiVersion:v1
-kind:ConfigMap
-metadata:
-# If you want to use this config map by default, name it "artifact-repositories". Otherwise, you can provide a reference to a
-# different config map in `artifactRepositoryRef.configMap`.
-name:artifact-repositories
-annotations:
-# v3.0 and after - if you want to use a specific key, put that key into this annotation.
-workflows.argoproj.io/default-artifact-repository:default-oss-artifact-repository
-data:
-default-oss-artifact-repository:|
-oss:
-endpoint: http://oss-cn-zhangjiakou-internal.aliyuncs.com
-bucket: $mybucket
-useSDKCreds: true
-
Create an Azure Storage account and a container within that account. There are a number of
-ways to accomplish this, including the Azure Portal or the
-CLI.
-
-
Retrieve the blob service endpoint for the storage account. For example:
Configure azure artifact as following in the yaml.
-
-
artifacts:
--name:message
-path:/tmp/message
-azure:
-endpoint:https://mystorageaccountname.blob.core.windows.net
-container:my-container-name
-blob:path/in/container
-# accountKeySecret is a secret selector.
-# It references the k8s secret named 'my-azure-storage-credentials'.
-# This secret is expected to have have the key 'account-access-key',
-# containing the base64 encoded credentials to the storage account.
-#
-# If a managed identity has been assigned to the machines running the
-# workflow (e.g., https://docs.microsoft.com/en-us/azure/aks/use-managed-identity)
-# then accountKeySecret is not needed, and useSDKCreds should be
-# set to true instead:
-# useSDKCreds: true
-accountKeySecret:
-name:my-azure-storage-credentials
-key:account-access-key
-
-
If useSDKCreds is set to true, then the accountKeySecret value is not
-used and authentication with Azure will be attempted using a
-DefaultAzureCredential
-instead.
In order for Argo to use your artifact repository, you can configure it as the
-default repository. Edit the workflow-controller config map with the correct
-endpoint and access/secret keys for your repository.
-
S3 compatible artifact repository bucket (such as AWS, GCS, MinIO, and Alibaba Cloud OSS)¶
The key is name of the object in the bucket The accessKeySecret and
-secretKeySecret are secret selectors that reference the specified kubernetes
-secret. The secret is expected to have the keys accessKey and secretKey,
-containing the base64 encoded credentials to the bucket.
-
For AWS, the accessKeySecret and secretKeySecret correspond to
-AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY respectively.
-
EC2 provides a meta-data API via which applications using the AWS SDK may assume
-IAM roles associated with the instance. If you are running argo on EC2 and the
-instance role allows access to your S3 bucket, you can configure the workflow
-step pods to assume the role. To do so, simply omit the accessKeySecret and
-secretKeySecret fields.
-
For GCS, the accessKeySecret and secretKeySecret for S3 compatible access
-can be obtained from the GCP Console. Note that S3 compatible access is on a per
-project rather than per bucket basis.
For MinIO, the accessKeySecret and secretKeySecret naturally correspond the
-AccessKey and SecretKey.
-
For Alibaba Cloud OSS, the accessKeySecret and secretKeySecret corresponds to
-accessKeyIDand accessKeySecret respectively.
-
Example:
-
$kubectleditconfigmapworkflow-controller-configmap-nargo# assumes argo was installed in the argo namespace
-...
-data:
-artifactRepository:|
-s3:
-bucket:my-bucket
-keyFormat:prefix/in/bucket#optional
-endpoint:my-minio-endpoint.default:9000#AWS => s3.amazonaws.com; GCS => storage.googleapis.com
-insecure:true#omit for S3/GCS. Needed when minio runs without TLS
-accessKeySecret:#omit if accessing via AWS IAM
-name:my-minio-cred
-key:accessKey
-secretKeySecret:#omit if accessing via AWS IAM
-name:my-minio-cred
-key:secretKey
-useSDKCreds:true#tells argo to use AWS SDK's default provider chain, enable for things like IRSA support
-
-
The secrets are retrieved from the namespace you use to run your workflows. Note
-that you can specify a keyFormat.
Argo also can use native GCS APIs to access a Google Cloud Storage bucket.
-
serviceAccountKeySecret references to a Kubernetes secret which stores a Google Cloud
-service account key to access the bucket.
-
Example:
-
$kubectleditconfigmapworkflow-controller-configmap-nargo# assumes argo was installed in the argo namespace
-...
-data:
-artifactRepository:|
-gcs:
-bucket:my-bucket
-keyFormat:prefix/in/bucket/{{workflow.name}}/{{pod.name}}#it should reference workflow variables, such as "{{workflow.name}}/{{pod.name}}"
-serviceAccountKeySecret:
-name:my-gcs-credentials
-key:serviceAccountKey
-
Argo can use native Azure APIs to access a Azure Blob Storage container.
-
accountKeySecret references to a Kubernetes secret which stores an Azure Blob
-Storage account shared key to access the container.
-
Example:
-
$kubectleditconfigmapworkflow-controller-configmap-nargo# assumes argo was installed in the argo namespace
-...
-data:
-artifactRepository:|
-azure:
-container:my-container
-blobNameFormat:prefix/in/container#optional, it could reference workflow variables, such as "{{workflow.name}}/{{pod.name}}"
-accountKeySecret:
-name:my-azure-storage-credentials
-key:account-access-key
-
This section shows how to access artifacts from non-default artifact
-repositories.
-
The endpoint, accessKeySecret and secretKeySecret are the same as for
-configuring the default artifact repository described previously.
-
templates:
--name:artifact-example
-inputs:
-artifacts:
--name:my-input-artifact
-path:/my-input-artifact
-s3:
-endpoint:s3.amazonaws.com
-bucket:my-aws-bucket-name
-key:path/in/bucket/my-input-artifact.tgz
-accessKeySecret:
-name:my-aws-s3-credentials
-key:accessKey
-secretKeySecret:
-name:my-aws-s3-credentials
-key:secretKey
-outputs:
-artifacts:
--name:my-output-artifact
-path:/my-output-artifact
-s3:
-endpoint:storage.googleapis.com
-bucket:my-gcs-bucket-name
-# NOTE that, by default, all output artifacts are automatically tarred and
-# gzipped before saving. So as a best practice, .tgz or .tar.gz
-# should be incorporated into the key name so the resulting file
-# has an accurate file extension.
-key:path/in/bucket/my-output-artifact.tgz
-accessKeySecret:
-name:my-gcs-s3-credentials
-key:accessKey
-secretKeySecret:
-name:my-gcs-s3-credentials
-key:secretKey
-region:my-GCS-storage-bucket-region
-container:
-image:debian:latest
-command:[sh,-c]
-args:["cp-r/my-input-artifact/my-output-artifact"]
-
With artifact streaming, artifacts don’t need to be saved to disk first. Artifact streaming is only supported in the following
-artifact drivers: S3 (v3.4+), Azure Blob (v3.4+), HTTP (v3.5+), and Artifactory (v3.5+).
-
Previously, when a user would click the button to download an artifact in the UI, the artifact would need to be written to the
-Argo Server’s disk first before downloading. If many users tried to download simultaneously, they would take up
-disk space and fail the download.
A container set templates is similar to a normal container or script template, but allows you to specify multiple
-containers to run within a single pod.
-
Because you have multiple containers within a pod, they will be scheduled on the same host. You can use cheap and fast
-empty-dir volumes instead of persistent volume claims to share data between steps.
It will use the sum total of all resource requests, maybe costing more than the same DAG template. This will be a problem if your requests already cost a lot. See below.
-
-
The containers can be arranged as a graph by specifying dependencies. This is suitable for running 10s rather than 100s
-of containers.
A container set actually starts all containers, and the Emissary only starts the main container process when the containers it depends on have completed. This mean that even though the container is doing no useful work, it is still consuming resources and you're still getting billed for them.
-
If your requests are small, this won't be a problem.
-
If your requests are large, set the resource requests so the sum total is the most you'll need at once.
-
Example A: a simple sequence e.g. a -> b -> c
-
-
a needs 1Gi memory
-
b needs 2Gi memory
-
c needs 1Gi memory
-
-
Then you know you need only a maximum of 2Gi. You could set as follows:
-
-
a requests 512Mi memory
-
b requests 1Gi memory
-
c requests 512Mi memory
-
-
The total is 2Gi, which is enough for b. We're all good.
-
Example B: Diamond DAG e.g. a diamond a -> b -> d and a -> c -> d, i.e. b and c run at the same time.
-
-
a needs 1000 cpu
-
b needs 2000 cpu
-
c needs 1000 cpu
-
d needs 1000 cpu
-
-
I know that b and c will run at the same time. So I need to make sure the total is 3000.
-
-
a requests 500 cpu
-
b requests 1000 cpu
-
c requests 1000 cpu
-
d requests 500 cpu
-
-
The total is 3000, which is enough for b + c. We're all good.
-
Example B: Lopsided requests, e.g. a -> b where a is cheap and b is expensive
-
-
a needs 100 cpu, 1Mi memory, runs for 10h
-
b needs 8Ki GPU, 100 Gi memory, 200 Ki GPU, runs for 5m
-
-
Can you see the problem here? a only has small requests, but the container set will use the total of all requests. So it's as if you're using all that GPU for 10h. This will be expensive.
-
Solution: do not use container set when you have lopsided requests.
Suitable if you are running a workflow with many homogeneous pods.
-
-
Resource duration shows the amount of CPU and memory requested by a pod and is indicative of the cost. You can use this to find costly steps within your workflow.
Consider trying Volume Claim Templates or Volumes instead of Artifacts¶
-
-
Suitable if you have a workflow that passes a lot of artifacts within itself.
-
-
Copying artifacts to and from storage outside of a cluster can be expensive. The correct choice is dependent on what your artifact storage provider is vs. what volume they are using. For example, we believe it may be more expensive to allocate and delete a new block storage volume (AWS EBS, GCP persistent disk) every workflow using the PVC feature, than it is to upload and download some small files to object storage (AWS S3, GCP cloud storage).
-
On the other hand if you are using a NFS volume shared between all your workflows with large artifacts, that might be cheaper than the data transfer and storage costs of object storage.
-
Consider:
-
-
Data transfer costs (upload/download vs. copying)
-
Data storage costs (object storage vs. volume)
-
Requirement for parallel access to data (NFS vs. block storage vs. artifact)
-
-
When using volume claims, consider configuring Volume Claim GC. By default, claims are only deleted when a workflow is successful.
A workflow (and for that matter, any Kubernetes resource) will incur a cost as long as it exists in your cluster, even after it's no longer running.
-
The workflow controller memory and CPU needs to increase linearly with the number of pods and workflows you are currently running.
-
You should delete workflows once they are no longer needed.
-You can enable the Workflow Archive to continue viewing them after they are removed from Kubernetes.
-
Limit the total number of workflows using:
-
-
Active Deadline Seconds - terminate running workflows that do not complete in a set time. This will make sure workflows do not run forever.
spec:
-# must complete in 8h (28,800 seconds)
-activeDeadlineSeconds:28800
-# keep workflows for 1d (86,400 seconds)
-ttlStrategy:
-secondsAfterCompletion:86400
-# delete all pods as soon as they complete
-podGC:
-strategy:OnPodCompletion
-
This above limit is suitable for the Argo Server, as this is stateless. The Workflow Controller is stateful and will scale to the number of live workflows - so you are likely to need higher values.
CronWorkflow are workflows that run on a preset schedule. They are designed to be converted from Workflow easily and to mimic the same options as Kubernetes CronJob. In essence, CronWorkflow = Workflow + some specific cron options.
CronWorkflow.spec.workflowSpec is the same type as Workflow.spec and serves as a template for Workflow objects that are created from it. Everything under this spec will be converted to a Workflow.
-
The resulting Workflow name will be a generated name based on the CronWorkflow name. In this example it could be something like test-cron-wf-tj6fe.
-
CronWorkflow.spec.workflowMetadata can be used to add labels and annotations.
Schedule at which the Workflow will be run. E.g. 5 4 * * *
-
-
-
timezone
-
Machine timezone
-
Timezone during which the Workflow will be run from the IANA timezone standard, e.g. America/Los_Angeles
-
-
-
suspend
-
false
-
If true Workflow scheduling will not occur. Can be set from the CLI, GitOps, or directly
-
-
-
concurrencyPolicy
-
Allow
-
Policy that determines what to do if multiple Workflows are scheduled at the same time. Available options: Allow: allow all, Replace: remove all old before scheduling a new, Forbid: do not allow any new while there are old
-
-
-
startingDeadlineSeconds
-
0
-
Number of seconds after the last successful run during which a missed Workflow will be run
-
-
-
successfulJobsHistoryLimit
-
3
-
Number of successful Workflows that will be persisted at a time
-
-
-
failedJobsHistoryLimit
-
1
-
Number of failed Workflows that will be persisted at a time
If the workflow-controller crashes (and hence the CronWorkflow controller), there are some options you can set to ensure that CronWorkflows that would have been scheduled while the controller was down can still run. Mainly startingDeadlineSeconds can be set to specify the maximum number of seconds past the last successful run of a CronWorkflow during which a missed run will still be executed.
-
For example, if a CronWorkflow that runs every minute is last run at 12:05:00, and the controller crashes between 12:05:55 and 12:06:05, then the expected execution time of 12:06:00 would be missed. However, if startingDeadlineSeconds is set to a value greater than 65 (the amount of time passing between the last scheduled run time of 12:05:00 and the current controller restart time of 12:06:05), then a single instance of the CronWorkflow will be executed exactly at 12:06:05.
-
Currently only a single instance will be executed as a result of setting startingDeadlineSeconds.
-
This setting can also be configured in tandem with concurrencyPolicy to achieve more fine-tuned control.
Daylight Saving (DST) is taken into account when using timezone. This means that, depending on the local time of the scheduled job, argo will schedule the workflow once, twice, or not at all when the clock moves forward or back.
-
For example, with timezone set at America/Los_Angeles, we have daylight saving
-
-
-
+1 hour (DST start) at 2020-03-08 02:00:00:
-
Note: The schedules between 02:00 a.m. to 02:59 a.m. were skipped on Mar 8th due to the clock being moved forward:
-
-
-
-
cron
-
sequence
-
workflow execution time
-
-
-
-
-
59 1 ** *
-
1
-
2020-03-08 01:59:00 -0800 PST
-
-
-
-
2
-
2020-03-09 01:59:00 -0700 PDT
-
-
-
-
3
-
2020-03-10 01:59:00 -0700 PDT
-
-
-
0 2 ** *
-
1
-
2020-03-09 02:00:00 -0700 PDT
-
-
-
-
2
-
2020-03-10 02:00:00 -0700 PDT
-
-
-
-
3
-
2020-03-11 02:00:00 -0700 PDT
-
-
-
1 2 ** *
-
1
-
2020-03-09 02:01:00 -0700 PDT
-
-
-
-
2
-
2020-03-10 02:01:00 -0700 PDT
-
-
-
-
3
-
2020-03-11 02:01:00 -0700 PDT
-
-
-
-
-
-
-1 hour (DST end) at 2020-11-01 02:00:00:
-
Note: the schedules between 01:00 a.m. to 01:59 a.m. were triggered twice on Nov 1st due to the clock being set back:
We have intentionally made this feature available with only bare-bones functionality. Our hope is that we are able to build this feature with our community's feedback. If you have ideas and use cases for this feature, please open an enhancement proposal on GitHub.
-
Additionally, please take a look at our current ideas at the bottom of this document.
Users often source and transform data as part of their workflows. The data template provides first-class support for these common operations.
-
data templates can best be understood by looking at a common data sourcing and transformation operation in bash:
-
find-r.|grep".pdf"|sed"s/foo/foo.ready/"
-
-
Such operations consist of two main parts:
-
-
A "source" of data: find -r .
-
A series of "transformations" which transform the output of the source serially: | grep ".pdf" | sed "s/foo/foo.ready/"
-
-
This operation, for example, could be useful in sourcing a potential list of files to be processed and filtering and manipulating the list as desired.
-
In Argo, this operation would be written as:
-
-name:generate-artifacts
-data:
-source:# Define a source for the data, only a single "source" is permitted
-artifactPaths:# A predefined source: Generate a list of all artifact paths in a given repository
-s3:# Source from an S3 bucket
-bucket:test
-endpoint:minio:9000
-insecure:true
-accessKeySecret:
-name:my-minio-cred
-key:accesskey
-secretKeySecret:
-name:my-minio-cred
-key:secretkey
-transformation:# The source is then passed to be transformed by transformations defined here
--expression:"filter(data,{#endsWith\".pdf\"})"
--expression:"map(data,{#+\".ready\"})"
-
A data template must always contain a source. Current available sources:
-
-
artifactPaths: generates a list of artifact paths from the artifact repository specified
-
-
A data template may contain any number of transformations (or zero). The transformations will be applied serially in order. Current available transformations:
-
-
-
expression: an expr expression. See language definition here. When defining expr expressions Argo will pass the available data to the environment as a variable called data (see example above).
-
We understand that the expression transformation is limited. We intend to greatly expand the functionality of this template with our community's feedback. Please see the link at the top of this document to submit ideas or use cases for this feature.
The debug pause feature makes it possible to pause individual workflow steps for debugging before, after or both and then release the steps from the paused state. Currently this feature is only supported when using the Emissary Executor
-
In order to pause a container env variables are used:
-
-
ARGO_DEBUG_PAUSE_AFTER - to pause a step after execution
-
ARGO_DEBUG_PAUSE_BEFORE - to pause a step before execution
In order to release a step from a pause state, marker files are used named /var/run/argo/ctr/main/after or /var/run/argo/ctr/main/before corresponding to when the step is paused. Pausing steps can be used together with ephemeral containers when a shell is not available in the used container.
1) Create a workflow where the debug pause env in set, in this example ARGO_DEBUG_PAUSE_AFTER will be set and thus the step will be paused after execution of the user code.
In order to have access to the persistence volume used by the workflow step, --share-processes will have to be used.
-
The ephemeral container can be used to perform debugging operations. When debugging has been completed, create the marker file to allow the workflow step to continue. When using process name space sharing container file systems are visible to other containers in the pod through the /proc/$pid/root link.
Default Workflow spec values can be set at the controller config map that will apply to all Workflows executed from said controller.
-If a Workflow has a value that also has a default value set in the config map, the Workflow's value will take precedence.
Default Workflow values can be specified by adding them under the workflowDefaults key in the workflow-controller-configmap.
-Values can be added as they would under the Workflow.spec tag.
-
For example, to specify default values that would partially produce the following Workflow:
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:gc-ttl-
-annotations:
-argo:workflows
-labels:
-foo:bar
-spec:
-ttlStrategy:
-secondsAfterSuccess:5# Time to live after workflow is successful
-parallelism:3
-
-
The following would be specified in the Config Map:
-
# This file describes the config settings available in the workflow controller configmap
-apiVersion:v1
-kind:ConfigMap
-metadata:
-name:workflow-controller-configmap
-data:
-# Default values that will apply to all Workflows from this controller, unless overridden on the Workflow-level
-workflowDefaults:|
-metadata:
-annotations:
-argo: workflows
-labels:
-foo: bar
-spec:
-ttlStrategy:
-secondsAfterSuccess: 5
-parallelism: 3
-
While by default, the Docker and PNS workflow executors can get output artifacts/parameters from the base layer (e.g. /tmp), neither the Kubelet nor the K8SAPI executors can. It is unlikely you can get output artifacts/parameters from the base layer if you run your workflow pods with a security context.
-
You can work-around this constraint by mounting volumes onto your pod. The easiest way to do this is to use as emptyDir volume.
-
-
Note
-
This is only needed for output artifacts/parameters. Input artifacts/parameters are automatically mounted to an empty-dir if needed
Previous to version 2.8, the only way to specify dependencies in DAG templates was to use the dependencies field and
-specify a list of other tasks the current task depends on. This syntax was limiting because it does not allow the user to
-specify which result of the task to depend on. For example, a task may only be relevant to run if the dependent task
-succeeded (or failed, etc.).
To remedy this, there exists a new field called depends, which allows users to specify dependent tasks, their statuses,
-as well as any complex boolean logic. The field is a string field and the syntax is expression-like with operands having
-form <task-name>.<task-result>. Examples include task-1.Succeeded, task-2.Failed, task-3.Daemoned. The full list of
-available task results is as follows:
-
-
-
-
Task Result
-
Description
-
Meaning
-
-
-
-
-
.Succeeded
-
Task Succeeded
-
Task finished with no error
-
-
-
.Failed
-
Task Failed
-
Task exited with a non-0 exit code
-
-
-
.Errored
-
Task Errored
-
Task had an error other than a non-0 exit code
-
-
-
.Skipped
-
Task Skipped
-
Task was skipped
-
-
-
.Omitted
-
Task Omitted
-
Task was omitted
-
-
-
.Daemoned
-
Task is Daemoned and is not Pending
-
-
-
-
-
For convenience, if an omitted task result is equivalent to (task.Succeeded || task.Skipped || task.Daemoned).
In the case that you're depending on a task that uses withItems, you can depend on
-whether any of the item tasks are successful or all have failed using .AnySucceeded and .AllFailed, for example:
-
depends:"task-1.AnySucceeded||task-2.AllFailed"
-
-
Compatibility with dependencies and dag.task.continueOn¶
-
This feature is fully compatible with dependencies and conversion is easy.
-
To convert simply join your dependencies with &&:
-
dependencies:["A","B","C"]
-
-
is equivalent to:
-
depends:"A&&B&&C"
-
-
Because of the added control found in depends, the dag.task.continueOn is not available when using it. Furthermore,
-it is not possible to use both dependencies and depends in the same task group.
This document outlines environment variables that can be used to customize behavior.
-
-
Warning
-
Environment variables are typically added to test out experimental features and should not be used by most users.
-Environment variables may be removed at any time.
Whether to consider all pod changes as significant during pod reconciliation.
-
-
-
ALWAYS_OFFLOAD_NODE_STATUS
-
bool
-
false
-
Whether to always offload the node status.
-
-
-
ARCHIVED_WORKFLOW_GC_PERIOD
-
time.Duration
-
24h
-
The periodicity for GC of archived workflows.
-
-
-
ARGO_PPROF
-
bool
-
false
-
Enable pprof endpoints
-
-
-
ARGO_PROGRESS_PATCH_TICK_DURATION
-
time.Duration
-
1m
-
How often self reported progress is patched into the pod annotations which means how long it takes until the controller picks up the progress change. Set to 0 to disable self reporting progress.
-
-
-
ARGO_PROGRESS_FILE_TICK_DURATION
-
time.Duration
-
3s
-
How often the progress file is read by the executor. Set to 0 to disable self reporting progress.
-
-
-
ARGO_REMOVE_PVC_PROTECTION_FINALIZER
-
bool
-
true
-
Remove the kubernetes.io/pvc-protection finalizer from persistent volume claims (PVC) after marking PVCs created for the workflow for deletion, so deleted is not blocked until the pods are deleted. #6629
-
-
-
ARGO_TRACE
-
string
-
``
-
Whether to enable tracing statements in Argo components.
-
-
-
ARGO_AGENT_PATCH_RATE
-
time.Duration
-
DEFAULT_REQUEUE_TIME
-
Rate that the Argo Agent will patch the workflow task-set.
-
-
-
ARGO_AGENT_CPU_LIMIT
-
resource.Quantity
-
100m
-
CPU resource limit for the agent.
-
-
-
ARGO_AGENT_MEMORY_LIMIT
-
resource.Quantity
-
256m
-
Memory resource limit for the agent.
-
-
-
BUBBLE_ENTRY_TEMPLATE_ERR
-
bool
-
true
-
Whether to bubble up template errors to workflow.
-
-
-
CACHE_GC_PERIOD
-
time.Duration
-
0s
-
How often to perform memoization cache GC, which is disabled by default and can be enabled by providing a non-zero duration.
-
-
-
CACHE_GC_AFTER_NOT_HIT_DURATION
-
time.Duration
-
30s
-
When a memoization cache has not been hit after this duration, it will be deleted.
-
-
-
CRON_SYNC_PERIOD
-
time.Duration
-
10s
-
How often to sync cron workflows.
-
-
-
DEFAULT_REQUEUE_TIME
-
time.Duration
-
10s
-
The re-queue time for the rate limiter of the workflow queue.
-
-
-
DISABLE_MAX_RECURSION
-
bool
-
false
-
Set to true to disable the recursion preventer, which will stop a workflow running which has called into a child template 100 times
-
-
-
EXPRESSION_TEMPLATES
-
bool
-
true
-
Escape hatch to disable expression templates.
-
-
-
EVENT_AGGREGATION_WITH_ANNOTATIONS
-
bool
-
false
-
Whether event annotations will be used when aggregating events.
-
-
-
GZIP_IMPLEMENTATION
-
string
-
PGZip
-
The implementation of compression/decompression. Currently only "PGZip" and "GZip" are supported.
-
-
-
INFORMER_WRITE_BACK
-
bool
-
true
-
Whether to write back to informer instead of catching up.
-
-
-
HEALTHZ_AGE
-
time.Duration
-
5m
-
How old a un-reconciled workflow is to report unhealthy.
-
-
-
INDEX_WORKFLOW_SEMAPHORE_KEYS
-
bool
-
true
-
Whether or not to index semaphores.
-
-
-
LEADER_ELECTION_IDENTITY
-
string
-
Controller's metadata.name
-
The ID used for workflow controllers to elect a leader.
-
-
-
LEADER_ELECTION_DISABLE
-
bool
-
false
-
Whether leader election should be disabled.
-
-
-
LEADER_ELECTION_LEASE_DURATION
-
time.Duration
-
15s
-
The duration that non-leader candidates will wait to force acquire leadership.
-
-
-
LEADER_ELECTION_RENEW_DEADLINE
-
time.Duration
-
10s
-
The duration that the acting master will retry refreshing leadership before giving up.
-
-
-
LEADER_ELECTION_RETRY_PERIOD
-
time.Duration
-
5s
-
The duration that the leader election clients should wait between tries of actions.
-
-
-
MAX_OPERATION_TIME
-
time.Duration
-
30s
-
The maximum time a workflow operation is allowed to run for before re-queuing the workflow onto the work queue.
-
-
-
OFFLOAD_NODE_STATUS_TTL
-
time.Duration
-
5m
-
The TTL to delete the offloaded node status. Currently only used for testing.
-
-
-
OPERATION_DURATION_METRIC_BUCKET_COUNT
-
int
-
6
-
The number of buckets to collect the metric for the operation duration.
-
-
-
POD_NAMES
-
string
-
v2
-
Whether to have pod names contain the template name (v2) or be the node id (v1) - should be set the same for Argo Server.
-
-
-
RECENTLY_STARTED_POD_DURATION
-
time.Duration
-
10s
-
The duration of a pod before the pod is considered to be recently started.
-
-
-
RETRY_BACKOFF_DURATION
-
time.Duration
-
10ms
-
The retry back-off duration when retrying API calls.
-
-
-
RETRY_BACKOFF_FACTOR
-
float
-
2.0
-
The retry back-off factor when retrying API calls.
-
-
-
RETRY_BACKOFF_STEPS
-
int
-
5
-
The retry back-off steps when retrying API calls.
-
-
-
RETRY_HOST_NAME_LABEL_KEY
-
string
-
kubernetes.io/hostname
-
The label key for host name used when retrying templates.
-
-
-
TRANSIENT_ERROR_PATTERN
-
string
-
""
-
The regular expression that represents additional patterns for transient errors.
-
-
-
WF_DEL_PROPAGATION_POLICY
-
string
-
""
-
The deletion propagation policy for workflows.
-
-
-
WORKFLOW_GC_PERIOD
-
time.Duration
-
5m
-
The periodicity for GC of workflows.
-
-
-
SEMAPHORE_NOTIFY_DELAY
-
time.Duration
-
1s
-
Tuning Delay when notifying semaphore waiters about availability in the semaphore
-
-
-
-
CLI parameters of the Controller can be specified as environment variables with the ARGO_ prefix.
-For example:
-
workflow-controller--managed-namespace=argo
-
-
Can be expressed as:
-
ARGO_MANAGED_NAMESPACE=argoworkflow-controller
-
-
You can set environment variables for the Controller Deployment's container spec like the following:
Disable the retrieval of the list of label values for keys based on this regular expression.
-
-
-
FIRST_TIME_USER_MODAL
-
bool
-
true
-
Show this modal.
-
-
-
FEEDBACK_MODAL
-
bool
-
true
-
Show this modal.
-
-
-
IP_KEY_FUNC_HEADERS
-
string
-
""
-
List of comma separated request headers containing IPs to use for rate limiting. For example, "X-Forwarded-For,X-Real-IP". By default, uses the request's remote IP address.
-
-
-
NEW_VERSION_MODAL
-
bool
-
true
-
Show this modal.
-
-
-
POD_NAMES
-
string
-
v2
-
Whether to have pod names contain the template name (v2) or be the node id (v1) - should be set the same for Controller
-
-
-
GRPC_MESSAGE_SIZE
-
string
-
104857600
-
Use different GRPC Max message size for Server (supporting huge workflows).
-
-
-
-
CLI parameters of the Server can be specified as environment variables with the ARGO_ prefix.
-For example:
-
argoserver--managed-namespace=argo
-
-
Can be expressed as:
-
ARGO_MANAGED_NAMESPACE=argoargoserver
-
-
You can set environment variables for the Server Deployment's container spec like the following:
The event endpoint will always return in under 10 seconds because the event will be queued and processed asynchronously. This means you will not be notified synchronously of failure. It will return a failure (503) if the event processing queue is full.
-
-
Processing Order
-
Events may not always be processed in the order they are received.
Before the binding between an event and a workflow template, you must create the workflow template that you want to trigger.
-The following one takes in input the "message" parameter specified into the API call body, passed through the WorkflowEventBinding parameters section, and finally resolved here as the message of the whalesay image.
A workflow template will be submitted (i.e. workflow created from it) and that can be created using parameters from the event itself.
-The following example will be triggered by an event with "message" in the payload. That message will be used as an argument for the created workflow. Note that the name of the meta-data header "x-argo-e2e" is lowercase in the selector to match. Incoming header names are converted to lowercase.
-
apiVersion:argoproj.io/v1alpha1
-kind:WorkflowEventBinding
-metadata:
-name:event-consumer
-spec:
-event:
-# metadata header name must be lowercase to match in selector
-selector:payload.message != "" && metadata["x-argo-e2e"] == ["true"] && discriminator == "my-discriminator"
-submit:
-workflowTemplateRef:
-name:my-wf-tmple
-arguments:
-parameters:
--name:message
-valueFrom:
-event:payload.message
-
-
Please, notice that workflowTemplateRef refers to a template with the name my-wf-tmple, this template has to be created before the triggering of the event.
-After that you have to apply the above explained WorkflowEventBinding (in this example this is called event-template.yml) to realize the binding between Workflow Template and event (you can use kubectl to do that):
-
kubectlapply-fevent-template.yml
-
-
Finally you can trigger the creation of your first parametrized workflow template, by using the following call:
You can customize the name of the submitted workflow as well as add annotations and
-labels. This is done by adding a metadata object to the submit object.
-
Normally the name of the workflow created from an event is simply the name of the
-template with a time-stamp appended. This can be customized by setting the name in the
-metadata object.
-
Annotations and labels are added in the same fashion.
-
All the values for the name, annotations and labels are treated as expressions (see
-below for details). The metadata object is the same metadata type as on all
-Kubernetes resources and as such is parsed in the same manner. It is best to enclose
-the expression in single quotes to avoid any problems when submitting the event
-binding to Kubernetes.
-
This is an example snippet of how to set the name, annotations and labels. This is
-based on the workflow binding from above, and the first event.
This will result in the workflow being named "hello-world" instead of
-my-wf-tmple-<timestamp>. There will be an extra label with the key someLabel and
-a value of "literal string". There will also be an extra annotation with the key
-anAnnotation and a value of "hello"
-
Be careful when setting the name. If the name expression evaluates to that of a currently
-existing workflow, the new workflow will fail to submit.
Because the endpoint accepts any JSON data, it is the user's responsibility to write a suitable expression to correctly filter the events they are interested in. Therefore, DO NOT assume the existence of any fields, and guard against them using a nil check.
This is only for edge-cases where neither the payload, or meta-data provide enough information to discriminate. Typically, it should be empty and ignored.
You MUST run a minimum of two Argo Server replicas if you do not want to lose events.
-
-
If you are processing large numbers of events, you may need to scale up the Argo Server to handle them. By default, a single Argo Server can be processing 64 events before the endpoint will start returning 503 errors.
-
Vertically you can:
-
-
Increase the size of the event operation queue --event-operation-queue-size (good for temporary event bursts).
-
Increase the number of workers --event-worker-count (good for sustained numbers of events).
-
-
Horizontally you can:
-
-
Run more Argo Servers (good for sustained numbers of events AND high-availability).
Tip: The port number can be anything, but must not conflict with other plugins. Don't use common ports such as 80,
-443, 8080, 8081, 8443. If you plan to publish your plugin, choose a random port number under 10,000 and create a PR to
-add your plugin. If not, use a port number greater than 10,000.
-
We'll need to create a script that starts a HTTP server. Save this as server.py:
Tip: Plugins can be written in any language you can run as a container. Python is convenient because you can embed
-the script in the container.
-
Some things to note here:
-
-
You only need to implement the calls you need. Return 404 and it won't be called again.
-
The path is the RPC method name.
-
You should check that the Authorization header contains the same value as /var/run/argo/token. Return 403 if not
-
The request body contains the template's input parameters.
-
The response body may contain the node's result, including the phase (e.g. "Succeeded" or "Failed") and a message.
-
If the response is {}, then the plugin is saying it cannot execute the plugin template, e.g. it is a Slack plugin,
- but the template is a Tekton job.
-
If the status code is 404, then the plugin will not be called again.
-
If you save the file as server.*, it will be copied to the sidecar container's args field. This is useful for building self-contained plugins in scripting languages like Python or Node.JS.
It might be the case that the plugin can't finish straight away. E.g. it starts a long running task. When that happens,
-you return "Pending" or "Running" a and a re-queue time:
An AWS EBS disk must exist before mounting to a container. The disk
-must also be in the same AWS zone as the kubelet. An AWS EBS disk
-can only be mounted as read/write once. AWS EBS volumes support
-ownership management and SELinux relabeling.
-
-
Properties
-
-
-
-
Name
-
Type
-
Go type
-
Required
-
Default
-
Description
-
Example
-
-
-
-
-
fsType
-
string
-
string
-
-
-
fsType is the filesystem type of the volume that you want to mount.
-
-
-
-
Tip: Ensure that the filesystem type is supported by the host operating system.
-
-
-
-
-
-
-
-
-
Examples: "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" if unspecified.
-
-
-
-
-
-
-
-
-
More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore
-
-
-
-
-
-
-
-
-
TODO: how do we prevent errors in the filesystem from compromising the machine
-
-
-
-
-
-
-
-
-
+optional
-
-
-
-
-
-
-
-
-
partition
-
int32 (formatted integer)
-
int32
-
-
-
partition is the partition in the volume that you want to mount.
-
-
-
-
If omitted, the default is to mount by volume name.
-
-
-
-
-
-
-
-
-
Examples: For volume /dev/sda1, you specify the partition as "1".
-
-
-
-
-
-
-
-
-
Similarly, the volume partition for /dev/sda is "0" (or you can leave the property empty).
-
-
-
-
-
-
-
-
-
+optional
-
-
-
-
-
-
-
-
-
readOnly
-
boolean
-
bool
-
-
-
readOnly value true will force the readOnly setting in VolumeMounts.
-
-
-
-
More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore
-
-
-
-
-
-
-
-
-
+optional
-
-
-
-
-
-
-
-
-
volumeID
-
string
-
string
-
-
-
volumeID is unique ID of the persistent disk resource in AWS (Amazon EBS volume).
-
-
-
-
More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore
It will unmarshall int64, int32, float64, float32, boolean, a plain string and represents it as string.
-It will marshall back to string - marshalling is not symmetric.
-
-
-
-
-
Name
-
Type
-
Go type
-
Default
-
Description
-
Example
-
-
-
-
-
AnyString
-
string
-
string
-
-
It will unmarshall int64, int32, float64, float32, boolean, a plain string and represents it as string.
-
-
-
-
It will marshall back to string - marshalling is not symmetric.
ArtifactGC describes how to delete artifacts from completed Workflows - this is embedded into the WorkflowLevelArtifactGC, and also used for individual Artifacts to override that as needed
It is used as single artifact in the context of inputs/outputs (e.g. outputs.artifacts.artname).
-It is also used to describe the location of multiple artifacts such as the archive location
-of a single workflow step, which the executor will use as a default location to store its files.
-
-
Properties
-
-
-
-
Name
-
Type
-
Go type
-
Required
-
Default
-
Description
-
Example
-
-
-
-
-
archiveLogs
-
boolean
-
bool
-
-
-
ArchiveLogs indicates if the container logs should be archived
A Cinder volume must exist before mounting to a container.
-The volume must also be in the same region as the kubelet.
-Cinder volumes support ownership management and SELinux relabeling.
-
-
Properties
-
-
-
-
Name
-
Type
-
Go type
-
Required
-
Default
-
Description
-
Example
-
-
-
-
-
fsType
-
string
-
string
-
-
-
fsType is the filesystem type to mount.
-
-
-
-
Must be a filesystem type supported by the host operating system.
-
-
-
-
-
-
-
-
-
Examples: "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" if unspecified.
-
-
-
-
-
-
-
-
-
More info: https://examples.k8s.io/mysql-cinder-pd/README.md
-
-
-
-
-
-
-
-
-
+optional
-
-
-
-
-
-
-
-
-
readOnly
-
boolean
-
bool
-
-
-
readOnly defaults to false (read/write). ReadOnly here will force
-
-
-
-
the ReadOnly setting in VolumeMounts.
-
-
-
-
-
-
-
-
-
More info: https://examples.k8s.io/mysql-cinder-pd/README.md
The contents of the target ConfigMap's Data field will be presented in a
-projected volume as files using the keys in the Data field as the file names,
-unless the items element is populated with specific mappings of keys to paths.
-Note that this is identical to a configmap volume source without the default
-mode.
The contents of the target ConfigMap's Data field will be presented in a
-volume as files using the keys in the Data field as the file names, unless
-the items element is populated with specific mappings of keys to paths.
-ConfigMap volumes support ownership management and SELinux relabeling.
-
-
Properties
-
-
-
-
Name
-
Type
-
Go type
-
Required
-
Default
-
Description
-
Example
-
-
-
-
-
defaultMode
-
int32 (formatted integer)
-
int32
-
-
-
defaultMode is optional: mode bits used to set permissions on created files by default.
-
-
-
-
Must be an octal value between 0000 and 0777 or a decimal value between 0 and 511.
-
-
-
-
-
-
-
-
-
YAML accepts both octal and decimal values, JSON requires decimal values for mode bits.
-
-
-
-
-
-
-
-
-
Defaults to 0644.
-
-
-
-
-
-
-
-
-
Directories within the path are not affected by this setting.
-
-
-
-
-
-
-
-
-
This might be in conflict with other options that affect the file
-
-
-
-
-
-
-
-
-
mode, like fsGroup, and the result can be other mode bits set.
Optional: mode bits used to set permissions on this file, must be an octal value
-
-
-
-
between 0000 and 0777 or a decimal value between 0 and 511.
-
-
-
-
-
-
-
-
-
YAML accepts both octal and decimal values, JSON requires decimal values for mode bits.
-
-
-
-
-
-
-
-
-
If not specified, the volume defaultMode will be used.
-
-
-
-
-
-
-
-
-
This might be in conflict with other options that affect the file
-
-
-
-
-
-
-
-
-
mode, like fsGroup, and the result can be other mode bits set.
-
-
-
-
-
-
-
-
-
+optional
-
-
-
-
-
-
-
-
-
path
-
string
-
string
-
-
-
Required: Path is the relative path name of the file to be created. Must not be absolute or contain the '..' path. Must be utf-8 encoded. The first item of the relative path must not start with '..'
Duration is a wrapper around time.Duration which supports correct
-marshaling to YAML and JSON. In particular, it marshals into strings, which
-can be used as map keys in json.
Each key is either a '.' representing the field itself, and will always map to an empty set,
-or a string representing a sub-field or item. The string will follow one of these four formats:
-'f:', where is the name of a field in a struct, or key in a map
-'v:', where is the exact json formatted value of a list item
-'i:', where is position of a item in a list
-'k:', where is a map of a list item's key fields to their unique values
-If a key maps to an empty Fields value, the field that key represents is part of the set.
-
-
The exact format is defined in sigs.k8s.io/structured-merge-diff
-+protobuf.options.(gogoproto.goproto_stringer)=false
A GCE PD must exist before mounting to a container. The disk must
-also be in the same GCE project and zone as the kubelet. A GCE PD
-can only be mounted as read/write once or read-only many times. GCE
-PDs support ownership management and SELinux relabeling.
-
-
Properties
-
-
-
-
Name
-
Type
-
Go type
-
Required
-
Default
-
Description
-
Example
-
-
-
-
-
fsType
-
string
-
string
-
-
-
fsType is filesystem type of the volume that you want to mount.
-
-
-
-
Tip: Ensure that the filesystem type is supported by the host operating system.
-
-
-
-
-
-
-
-
-
Examples: "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" if unspecified.
-
-
-
-
-
-
-
-
-
More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk
-
-
-
-
-
-
-
-
-
TODO: how do we prevent errors in the filesystem from compromising the machine
-
-
-
-
-
-
-
-
-
+optional
-
-
-
-
-
-
-
-
-
partition
-
int32 (formatted integer)
-
int32
-
-
-
partition is the partition in the volume that you want to mount.
-
-
-
-
If omitted, the default is to mount by volume name.
-
-
-
-
-
-
-
-
-
Examples: For volume /dev/sda1, you specify the partition as "1".
-
-
-
-
-
-
-
-
-
Similarly, the volume partition for /dev/sda is "0" (or you can leave the property empty).
-
-
-
-
-
-
-
-
-
More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk
-
-
-
-
-
-
-
-
-
+optional
-
-
-
-
-
-
-
-
-
pdName
-
string
-
string
-
-
-
pdName is unique name of the PD resource in GCE. Used to identify the disk in GCE.
-
-
-
-
More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk
-
-
-
-
-
-
-
-
-
readOnly
-
boolean
-
bool
-
-
-
readOnly here will force the ReadOnly setting in VolumeMounts.
-
-
-
-
Defaults to false.
-
-
-
-
-
-
-
-
-
More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk
DEPRECATED: GitRepo is deprecated. To provision a container with a git repo, mount an
-EmptyDir into an InitContainer that clones the repo using git, then mount the EmptyDir
-into the Pod's container.
-
-
Properties
-
-
-
-
Name
-
Type
-
Go type
-
Required
-
Default
-
Description
-
Example
-
-
-
-
-
directory
-
string
-
string
-
-
-
directory is the target directory name.
-
-
-
-
Must not contain or start with '..'. If '.' is supplied, the volume directory will be the
-
-
-
-
-
-
-
-
-
git repository. Otherwise, if specified, the volume will contain the git repository in
-
-
-
-
-
-
-
-
-
the subdirectory with the given name.
-
-
-
-
-
-
-
-
-
+optional
-
-
-
-
-
-
-
-
-
repository
-
string
-
string
-
-
-
repository is the URL
-
-
-
-
revision
-
string
-
string
-
-
-
revision is the commit hash for the specified revision.
A label selector is a label query over a set of resources. The result of matchLabels and
-matchExpressions are ANDed. An empty label selector matches all objects. A null
-label selector matches no objects.
-+structType=atomic
Lifecycle describes actions that the management system should take in response to container lifecycle
-events. For the PostStart and PreStop lifecycle handlers, management of the container blocks
-until the action is complete, unless the container process fails, in which case the handler is aborted.
LifecycleHandler defines a specific action that should be taken in a lifecycle
-hook. One and only one of the fields, except TCPSocket must be specified.
A node selector represents the union of the results of one or more label queries
-over a set of nodes; that is, it represents the OR of the selectors represented
-by the node selector terms.
-+structType=atomic
A null or empty node selector term matches no objects. The requirements of
-them are ANDed.
-The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.
-+structType=atomic
NoneStrategy indicates to skip tar process and upload the files or directory tree as independent
-files. Note that if the artifact is a directory, the artifact driver must support the ability to
-save/load the directory appropriately.
OwnerReference contains enough information to let you identify an owning
-object. An owning object must be in the same namespace as the dependent, or
-be cluster-scoped, so there is no namespace field.
-+structType=atomic
-
-
Properties
-
-
-
-
Name
-
Type
-
Go type
-
Required
-
Default
-
Description
-
Example
-
-
-
-
-
apiVersion
-
string
-
string
-
-
-
API version of the referent.
-
-
-
-
blockOwnerDeletion
-
boolean
-
bool
-
-
-
If true, AND if the owner has the "foregroundDeletion" finalizer, then
-
-
-
-
the owner cannot be deleted from the key-value store until this
-
-
-
-
-
-
-
-
-
reference is removed.
-
-
-
-
-
-
-
-
-
See https://kubernetes.io/docs/concepts/architecture/garbage-collection/#foreground-deletion
-
-
-
-
-
-
-
-
-
for how the garbage collector interacts with this field and enforces the foreground deletion.
-
-
-
-
-
-
-
-
-
Defaults to false.
-
-
-
-
-
-
-
-
-
To set this field, a user needs "delete" permission of the owner,
-
-
-
-
-
-
-
-
-
otherwise 422 (Unprocessable Entity) will be returned.
-
-
-
-
-
-
-
-
-
+optional
-
-
-
-
-
-
-
-
-
controller
-
boolean
-
bool
-
-
-
If true, this reference points to the managing controller.
-
-
-
-
+optional
-
-
-
-
-
-
-
-
-
kind
-
string
-
string
-
-
-
Kind of the referent.
-
-
-
-
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
-
-
-
-
-
-
-
-
-
name
-
string
-
string
-
-
-
Name of the referent.
-
-
-
-
More info: http://kubernetes.io/docs/user-guide/identifiers#names
PersistentVolumeClaimTemplate is used to produce
-PersistentVolumeClaim objects as part of an EphemeralVolumeSource.
-
-
Properties
-
-
-
-
Name
-
Type
-
Go type
-
Required
-
Default
-
Description
-
Example
-
-
-
-
-
annotations
-
map of string
-
map[string]string
-
-
-
Annotations is an unstructured key value map stored with a resource that may be
-
-
-
-
set by external tools to store and retrieve arbitrary metadata. They are not
-
-
-
-
-
-
-
-
-
queryable and should be preserved when modifying objects.
-
-
-
-
-
-
-
-
-
More info: http://kubernetes.io/docs/user-guide/annotations
-
-
-
-
-
-
-
-
-
+optional
-
-
-
-
-
-
-
-
-
clusterName
-
string
-
string
-
-
-
Deprecated: ClusterName is a legacy field that was always cleared by
-
-
-
-
the system and never used; it will be removed completely in 1.25.
-
-
-
-
-
-
-
-
-
-
The name in the go struct is changed to help clients detect
-accidental use.
-
+optional | |
-| creationTimestamp | Time| Time | | | | |
-| deletionGracePeriodSeconds | int64 (formatted integer)| int64 | | | Number of seconds allowed for this object to gracefully terminate before
-it will be removed from the system. Only set when deletionTimestamp is also set.
-May only be shortened.
-Read-only.
-+optional | |
-| deletionTimestamp | Time| Time | | | | |
-| finalizers | []string| []string | | | Must be empty before the object is deleted from the registry. Each entry
-is an identifier for the responsible component that will remove the entry
-from the list. If the deletionTimestamp of the object is non-nil, entries
-in this list can only be removed.
-Finalizers may be processed and removed in any order. Order is NOT enforced
-because it introduces significant risk of stuck finalizers.
-finalizers is a shared field, any actor with permission can reorder it.
-If the finalizer list is processed in order, then this can lead to a situation
-in which the component responsible for the first finalizer in the list is
-waiting for a signal (field value, external system, or other) produced by a
-component responsible for a finalizer later in the list, resulting in a deadlock.
-Without enforced ordering finalizers are free to order amongst themselves and
-are not vulnerable to ordering changes in the list.
-+optional
-+patchStrategy=merge | |
-| generateName | string| string | | | GenerateName is an optional prefix, used by the server, to generate a unique
-name ONLY IF the Name field has not been provided.
-If this field is used, the name returned to the client will be different
-than the name passed. This value will also be combined with a unique suffix.
-The provided value has the same validation rules as the Name field,
-and may be truncated by the length of the suffix required to make the value
-unique on the server.
-
If this field is specified and the generated name exists, the server will return a 409.
-
Applied only if Name is not specified.
-More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#idempotency
-+optional | |
-| generation | int64 (formatted integer)| int64 | | | A sequence number representing a specific generation of the desired state.
-Populated by the system. Read-only.
-+optional | |
-| labels | map of string| map[string]string | | | Map of string keys and values that can be used to organize and categorize
-(scope and select) objects. May match selectors of replication controllers
-and services.
-More info: http://kubernetes.io/docs/user-guide/labels
-+optional | |
-| managedFields | []ManagedFieldsEntry| []*ManagedFieldsEntry | | | ManagedFields maps workflow-id and version to the set of fields
-that are managed by that workflow. This is mostly for internal
-housekeeping, and users typically shouldn't need to set or
-understand this field. A workflow can be the user's name, a
-controller's name, or the name of a specific apply path like
-"ci-cd". The set of fields is always in the version that the
-workflow used when modifying the object.
-
+optional | |
-| name | string| string | | | Name must be unique within a namespace. Is required when creating resources, although
-some resources may allow a client to request the generation of an appropriate name
-automatically. Name is primarily intended for creation idempotence and configuration
-definition.
-Cannot be updated.
-More info: http://kubernetes.io/docs/user-guide/identifiers#names
-+optional | |
-| namespace | string| string | | | Namespace defines the space within which each name must be unique. An empty namespace is
-equivalent to the "default" namespace, but "default" is the canonical representation.
-Not all objects are required to be scoped to a namespace - the value of this field for
-those objects will be empty.
-
Must be a DNS_LABEL.
-Cannot be updated.
-More info: http://kubernetes.io/docs/user-guide/namespaces
-+optional | |
-| ownerReferences | []OwnerReference| []*OwnerReference | | | List of objects depended by this object. If ALL objects in the list have
-been deleted, this object will be garbage collected. If this object is managed by a controller,
-then an entry in this list will point to this controller, with the controller field set to true.
-There cannot be more than one managing controller.
-+optional
-+patchMergeKey=uid
-+patchStrategy=merge | |
-| resourceVersion | string| string | | | An opaque value that represents the internal version of this object that can
-be used by clients to determine when objects have changed. May be used for optimistic
-concurrency, change detection, and the watch operation on a resource or set of resources.
-Clients must treat these values as opaque and passed unmodified back to the server.
-They may only be valid for a particular resource or set of resources.
-
Populated by the system.
-Read-only.
-Value must be treated as opaque by clients and .
-More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency
-+optional | |
-| selfLink | string| string | | | Deprecated: selfLink is a legacy read-only field that is no longer populated by the system.
-+optional | |
-| spec | PersistentVolumeClaimSpec| PersistentVolumeClaimSpec | | | | |
-| uid | UID| UID | | | | |
This volume finds the bound PV and mounts that volume for the pod. A
-PersistentVolumeClaimVolumeSource is, essentially, a wrapper around another
-type of volume that is owned by someone else (the system).
-
-
Properties
-
-
-
-
Name
-
Type
-
Go type
-
Required
-
Default
-
Description
-
Example
-
-
-
-
-
claimName
-
string
-
string
-
-
-
claimName is the name of a PersistentVolumeClaim in the same namespace as the pod using this volume.
-
-
-
-
More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims
-
-
-
-
-
-
-
-
-
readOnly
-
boolean
-
bool
-
-
-
readOnly Will force the ReadOnly setting in VolumeMounts.
Defines a set of pods (namely those matching the labelSelector
-relative to the given namespace(s)) that this pod should be
-co-located (affinity) or not co-located (anti-affinity) with,
-where co-located is defined as running on a node whose value of
-the label with key matches that of any node on which
-a pod of the set of pods is running
Some fields are also present in container.securityContext. Field values of
-container.securityContext take precedence over field values of PodSecurityContext.
-
-
Properties
-
-
-
-
Name
-
Type
-
Go type
-
Required
-
Default
-
Description
-
Example
-
-
-
-
-
fsGroup
-
int64 (formatted integer)
-
int64
-
-
-
A special supplemental group that applies to all containers in a pod.
-
-
-
-
Some volume types allow the Kubelet to change the ownership of that volume
-
-
-
-
-
-
-
-
-
to be owned by the pod:
-
-
-
-
-
-
-
-
-
-
-
The owning GID will be the FSGroup
-
The setgid bit is set (new files created in the volume will be owned by FSGroup)
-
The permission bits are OR'd with rw-rw----
-
-
If unset, the Kubelet will not modify the ownership and permissions of any volume.
-Note that this field cannot be set when spec.os.name is windows.
-+optional | |
-| fsGroupChangePolicy | PodFSGroupChangePolicy| PodFSGroupChangePolicy | | | | |
-| runAsGroup | int64 (formatted integer)| int64 | | | The GID to run the entrypoint of the container process.
-Uses runtime default if unset.
-May also be set in SecurityContext. If set in both SecurityContext and
-PodSecurityContext, the value specified in SecurityContext takes precedence
-for that container.
-Note that this field cannot be set when spec.os.name is windows.
-+optional | |
-| runAsNonRoot | boolean| bool | | | Indicates that the container must run as a non-root user.
-If true, the Kubelet will validate the image at runtime to ensure that it
-does not run as UID 0 (root) and fail to start the container if it does.
-If unset or false, no such validation will be performed.
-May also be set in SecurityContext. If set in both SecurityContext and
-PodSecurityContext, the value specified in SecurityContext takes precedence.
-+optional | |
-| runAsUser | int64 (formatted integer)| int64 | | | The UID to run the entrypoint of the container process.
-Defaults to user specified in image metadata if unspecified.
-May also be set in SecurityContext. If set in both SecurityContext and
-PodSecurityContext, the value specified in SecurityContext takes precedence
-for that container.
-Note that this field cannot be set when spec.os.name is windows.
-+optional | |
-| seLinuxOptions | SELinuxOptions| SELinuxOptions | | | | |
-| seccompProfile | SeccompProfile| SeccompProfile | | | | |
-| supplementalGroups | []int64 (formatted integer)| []int64 | | | A list of groups applied to the first process run in each container, in addition
-to the container's primary GID. If unspecified, no groups will be added to
-any container.
-Note that this field cannot be set when spec.os.name is windows.
-+optional | |
-| sysctls | []Sysctl| []*Sysctl | | | Sysctls hold a list of namespaced sysctls used for the pod. Pods with unsupported
-sysctls (by the container runtime) might fail to launch.
-Note that this field cannot be set when spec.os.name is windows.
-+optional | |
-| windowsOptions | WindowsSecurityContextOptions| WindowsSecurityContextOptions | | | | |
An empty preferred scheduling term matches all objects with implicit weight 0
-(i.e. it's a no-op). A null preferred scheduling term matches no objects (i.e. is also a no-op).
::=
-(Note that may be empty, from the "" case in .)
- ::= 0 | 1 | ... | 9
- ::= |
- ::= | . | . | .
- ::= "+" | "-"
- ::= |
- ::= | |
- ::= Ki | Mi | Gi | Ti | Pi | Ei
-(International System of units; See: http://physics.nist.gov/cuu/Units/binary.html)
- ::= m | "" | k | M | G | T | P | E
-(Note that 1024 = 1Ki but 1000 = 1k; I didn't choose the capitalization.)
- ::= "e" | "E"
-
No matter which of the three exponent forms is used, no quantity may represent
-a number greater than 2^63-1 in magnitude, nor may it have more than 3 decimal
-places. Numbers larger or more precise will be capped or rounded up.
-(E.g.: 0.1m will rounded up to 1m.)
-This may be extended in the future if we require larger or smaller quantities.
-
When a Quantity is parsed from a string, it will remember the type of suffix
-it had, and will use the same type again when it is serialized.
-
Before serializing, Quantity will be put in "canonical form".
-This means that Exponent/suffix will be adjusted up or down (with a
-corresponding increase or decrease in Mantissa) such that:
-a. No precision is lost
-b. No fractional digits will be emitted
-c. The exponent (or suffix) is as large as possible.
-The sign will be omitted unless the number is negative.
-
Examples:
-1.5 will be serialized as "1500m"
-1.5Gi will be serialized as "1536Mi"
-
Note that the quantity will NEVER be internally represented by a
-floating point number. That is the whole point of this exercise.
-
Non-canonical values will still parse as long as they are well formed,
-but will be re-emitted in their canonical form. (So always use canonical
-form, or don't diff.)
-
This format is intended to make it difficult to use these numbers without
-writing some sort of special handling code in the hopes that that will
-cause implementors to also use a fixed point implementation.
S3EncryptionOptions used to determine encryption options during s3 operations
-
-
Properties
-
-
-
-
Name
-
Type
-
Go type
-
Required
-
Default
-
Description
-
Example
-
-
-
-
-
enableEncryption
-
boolean
-
bool
-
-
-
EnableEncryption tells the driver to encrypt objects if set to true. If kmsKeyId and serverSideCustomerKeySecret are not set, SSE-S3 will be used
-
-
-
-
kmsEncryptionContext
-
string
-
string
-
-
-
KmsEncryptionContext is a json blob that contains an encryption context. See https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html#encrypt_context for more information
-
-
-
-
kmsKeyId
-
string
-
string
-
-
-
KMSKeyId tells the driver to encrypt the object using the specified KMS Key.
The contents of the target Secret's Data field will be presented in a
-projected volume as files using the keys in the Data field as the file names.
-Note that this is identical to a secret volume source without the default
-mode.
The contents of the target Secret's Data field will be presented in a volume
-as files using the keys in the Data field as the file names.
-Secret volumes support ownership management and SELinux relabeling.
-
-
Properties
-
-
-
-
Name
-
Type
-
Go type
-
Required
-
Default
-
Description
-
Example
-
-
-
-
-
defaultMode
-
int32 (formatted integer)
-
int32
-
-
-
defaultMode is Optional: mode bits used to set permissions on created files by default.
-
-
-
-
Must be an octal value between 0000 and 0777 or a decimal value between 0 and 511.
-
-
-
-
-
-
-
-
-
YAML accepts both octal and decimal values, JSON requires decimal values
-
-
-
-
-
-
-
-
-
for mode bits. Defaults to 0644.
-
-
-
-
-
-
-
-
-
Directories within the path are not affected by this setting.
-
-
-
-
-
-
-
-
-
This might be in conflict with other options that affect the file
-
-
-
-
-
-
-
-
-
mode, like fsGroup, and the result can be other mode bits set.
ServiceAccountTokenProjection represents a projected service account token
-volume. This projection can be used to insert a service account token into
-the pods runtime filesystem for use against APIs (Kubernetes API Server or
-otherwise).
-
-
Properties
-
-
-
-
Name
-
Type
-
Go type
-
Required
-
Default
-
Description
-
Example
-
-
-
-
-
audience
-
string
-
string
-
-
-
audience is the intended audience of the token. A recipient of a token
-
-
-
-
must identify itself with an identifier specified in the audience of the
-
-
-
-
-
-
-
-
-
token, and otherwise should reject the token. The audience defaults to the
-
-
-
-
-
-
-
-
-
identifier of the apiserver.
-
-
-
-
-
-
-
-
-
+optional
-
-
-
-
-
-
-
-
-
expirationSeconds
-
int64 (formatted integer)
-
int64
-
-
-
expirationSeconds is the requested duration of validity of the service
-
-
-
-
account token. As the token approaches expiration, the kubelet volume
-
-
-
-
-
-
-
-
-
plugin will proactively rotate the service account token. The kubelet will
-
-
-
-
-
-
-
-
-
start trying to rotate the token if the token is older than 80 percent of
-
-
-
-
-
-
-
-
-
its time to live or if the token is older than 24 hours.Defaults to 1 hour
-
-
-
-
-
-
-
-
-
and must be at least 10 minutes.
-
-
-
-
-
-
-
-
-
+optional
-
-
-
-
-
-
-
-
-
path
-
string
-
string
-
-
-
path is the path relative to the mount point of the file to project the
UID is a type that holds unique ID values, including UUIDs. Because we
-don't ONLY use UUIDs, this is an alias to string. Being a type captures
-intent and helps make sure that UIDs and names do not get conflated.
-
-
-
-
-
Name
-
Type
-
Go type
-
Default
-
Description
-
Example
-
-
-
-
-
UID
-
string
-
string
-
-
UID is a type that holds unique ID values, including UUIDs. Because we
-
-
-
-
don't ONLY use UUIDs, this is an alias to string. Being a type captures
-
-
-
-
-
-
-
-
intent and helps make sure that UIDs and names do not get conflated.
"token not valid", "any bearer token is able to login in the UI or use the API"¶
-
You may not have configured Argo Server authentication correctly.
-
If you want SSO, try running with --auth-mode=sso.
-If you're using --auth-mode=client, make sure you have Bearer in front of the ServiceAccount Secret, as mentioned in Access Token.
APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.io.k8s.community/contributors/devel/sig-architecture/api-conventions.md#resources
-
-
-
kind
-
string
-
Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.io.k8s.community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.io.k8s.community/contributors/devel/sig-architecture/api-conventions.md#resources
-
-
-
kind
-
string
-
Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.io.k8s.community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.io.k8s.community/contributors/devel/sig-architecture/api-conventions.md#resources
-
-
-
kind
-
string
-
Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.io.k8s.community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
Optional duration in seconds relative to the workflow start time which the workflow is allowed to run before the controller terminates the io.argoproj.workflow.v1alpha1. A value of zero is used to terminate a Running workflow
Arguments contain the parameters and artifacts sent to the workflow entrypoint Parameters are referencable globally using the 'workflow' variable prefix. e.g. {{io.argoproj.workflow.v1alpha1.parameters.myparam}}
ArtifactGC describes the strategy to use when deleting artifacts from completed or deleted workflows (applies to all output Artifacts unless Artifact.ArtifactGC is specified, which overrides this)
ArtifactRepositoryRef specifies the configMap name and key containing the artifact repository config.
-
-
-
automountServiceAccountToken
-
boolean
-
AutomountServiceAccountToken indicates whether a service account token should be automatically mounted in pods. ServiceAccountName of ExecutorConfig must be specified if this value is false.
PodDNSConfig defines the DNS parameters of a pod in addition to those generated from DNSPolicy.
-
-
-
dnsPolicy
-
string
-
Set DNS policy for the pod. Defaults to "ClusterFirst". Valid values are 'ClusterFirstWithHostNet', 'ClusterFirst', 'Default' or 'None'. DNS parameters given in DNSConfig will be merged with the policy selected with DNSPolicy. To have DNS options set along with hostNetwork, you have to specify DNS policy explicitly to 'ClusterFirstWithHostNet'.
-
-
-
entrypoint
-
string
-
Entrypoint is a template reference to the starting point of the io.argoproj.workflow.v1alpha1.
ImagePullSecrets is a list of references to secrets in the same namespace to use for pulling any images in pods that reference this ServiceAccount. ImagePullSecrets are distinct from Secrets because Secrets can be mounted in the pod, but ImagePullSecrets are only accessed by the kubelet. More info: https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod
Metrics are a list of metrics emitted from this Workflow
-
-
-
nodeSelector
-
Map< string , string >
-
NodeSelector is a selector which will result in all pods of the workflow to be scheduled on the selected node(s). This is able to be overridden by a nodeSelector specified in the template.
-
-
-
onExit
-
string
-
OnExit is a template reference which is invoked at the end of the workflow, irrespective of the success, failure, or error of the primary io.argoproj.workflow.v1alpha1.
-
-
-
parallelism
-
integer
-
Parallelism limits the max total parallel pods that can execute at the same time in a workflow
PodDisruptionBudget holds the number of concurrent disruptions that you allow for Workflow's Pods. Controller will automatically add the selector with workflow name, if selector is empty. Optional: Defaults to empty.
PodMetadata defines additional metadata that should be applied to workflow pods
-
-
-
~~podPriority~~
-
~~integer~~
-
~~Priority to apply to workflow pods.~~ DEPRECATED: Use PodPriorityClassName instead.
-
-
-
podPriorityClassName
-
string
-
PriorityClassName to apply to workflow pods.
-
-
-
podSpecPatch
-
string
-
PodSpecPatch holds strategic merge patch to apply against the pod spec. Allows parameterization of container fields which are not strings (e.g. resource limits).
-
-
-
priority
-
integer
-
Priority is used if controller is configured to process limited number of workflows in parallel. Workflows with higher priority are processed first.
RetryStrategy for all templates in the io.argoproj.workflow.v1alpha1.
-
-
-
schedulerName
-
string
-
Set scheduler name for all pods. Will be overridden if container/script template's scheduler name is set. Default scheduler will be used if neither specified.
SecurityContext holds pod-level security attributes and common container settings. Optional: Defaults to empty. See type description for default values of each field.
-
-
-
serviceAccountName
-
string
-
ServiceAccountName is the name of the ServiceAccount to run all pods of the workflow as.
-
-
-
shutdown
-
string
-
Shutdown will shutdown the workflow according to its ShutdownStrategy
-
-
-
suspend
-
boolean
-
Suspend will suspend the workflow and prevent execution of any future steps in the workflow
TTLStrategy limits the lifetime of a Workflow that has finished execution depending on if it Succeeded or Failed. If this struct is set, once the Workflow finishes, it will be deleted after the time to live expires. If this field is unset, the controller config map will hold the default values.
VolumeClaimTemplates is a list of claims that containers are allowed to reference. The Workflow controller will create the claims at the beginning of the workflow and delete the claims upon completion of the workflow
Nodes is a mapping between a node ID and the node's status.
-
-
-
offloadNodeStatusVersion
-
string
-
Whether on not node status has been offloaded to a database. If exists, then Nodes and CompressedNodes will be empty. This will actually be populated with a hash of the offloaded data.
PersistentVolumeClaims tracks all PVCs that were created as part of the io.argoproj.workflow.v1alpha1. The contents of this list are drained at the end of the workflow.
-
-
-
phase
-
string
-
Phase a simple, high-level summary of where the workflow is in its lifecycle. Will be "" (Unknown), "Pending", or "Running" before the workflow is completed, and "Succeeded", "Failed" or "Error" once the workflow has completed.
ConcurrencyPolicy is the K8s-style concurrency policy that will be used
-
-
-
failedJobsHistoryLimit
-
integer
-
FailedJobsHistoryLimit is the number of failed jobs to be kept at a time
-
-
-
schedule
-
string
-
Schedule is a schedule to run the Workflow in Cron format
-
-
-
startingDeadlineSeconds
-
integer
-
StartingDeadlineSeconds is the K8s-style deadline that will limit the time a CronWorkflow will be run after its original scheduled time if it is missed.
-
-
-
successfulJobsHistoryLimit
-
integer
-
SuccessfulJobsHistoryLimit is the number of successful jobs to be kept at a time
-
-
-
suspend
-
boolean
-
Suspend is a flag that will stop new CronWorkflows from running if set to true
-
-
-
timezone
-
string
-
Timezone is the timezone against which the cron schedule will be calculated, e.g. "Asia/Tokyo". Default is machine's local time.
Expression is a condition expression for when a node will be retried. If it evaluates to false, the node will not be retried and the retry strategy will be ignored
-
-
-
template
-
string
-
Template is the name of the template to execute by the hook
Expression is a condition expression for when a node will be retried. If it evaluates to false, the node will not be retried and the retry strategy will be ignored
Limit is the maximum number of retry attempts when retrying a container. It does not include the original container; the maximum number of total attempts will be limit + 1.
-
-
-
retryPolicy
-
string
-
RetryPolicy is a policy of NodePhase statuses that will be retried
Optional duration in seconds relative to the StartTime that the pod may be active on a node before the system actively tries to terminate the pod; value must be positive integer This field is only applicable to container and script templates.
Location in which all files related to the step will be stored (logs, artifacts, etc...). Can be overridden by individual items in Outputs. If omitted, will use the default artifact repository location configured in the controller, appended with the / in the key.
-
-
-
automountServiceAccountToken
-
boolean
-
AutomountServiceAccountToken indicates whether a service account token should be automatically mounted in pods. ServiceAccountName of ExecutorConfig must be specified if this value is false.
Executor holds configurations of the executor container.
-
-
-
failFast
-
boolean
-
FailFast, if specified, will fail this template if any of its child pods has failed. This is useful for when this template is expanded with withItems, etc.
Outputs describe the parameters and artifacts that this template produces
-
-
-
parallelism
-
integer
-
Parallelism limits the max total parallel pods that can execute at the same time within the boundaries of this template invocation. If additional steps/dag templates are invoked, the pods created by those templates will not be counted towards this total.
PodSpecPatch holds strategic merge patch to apply against the pod spec. Allows parameterization of container fields which are not strings (e.g. resource limits).
RetryStrategy describes how to retry a template when it fails
-
-
-
schedulerName
-
string
-
If specified, the pod will be dispatched by specified scheduler. Or it will be dispatched by workflow scope scheduler if specified. If neither specified, the pod will be dispatched by default scheduler.
SecurityContext holds pod-level security attributes and common container settings. Optional: Defaults to empty. See type description for default values of each field.
Synchronization holds synchronization lock configuration for this template
-
-
-
timeout
-
string
-
Timeout allows to set the total node execution timeout duration counting from the node's start time. This duration also includes time in which the node spends in Pending state. This duration may not be applied to Step or DAG templates.
NodeFlag tracks some history of node. e.g.) hooked, retried, etc.
-
-
-
outboundNodes
-
Array< string >
-
OutboundNodes tracks the node IDs which are considered "outbound" nodes to a template invocation. For every invocation of a template, there are nodes which we considered as "outbound". Essentially, these are last nodes in the execution sequence to run, before the template is considered completed. These nodes are then connected as parents to a following step. In the case of single pod steps (i.e. container, script, resource templates), this list will be nil since the pod itself is already considered the "outbound" node. In the case of DAGs, outbound nodes are the "target" tasks (tasks with no children). In the case of steps, outbound nodes are all the containers involved in the last step group. NOTE: since templates are composable, the list of outbound nodes are carried upwards when a DAG/steps template invokes another DAG/steps template. In other words, the outbound nodes of a template, will be a superset of the outbound nodes of its last children.
Outputs captures output parameter values and artifact locations produced by this template invocation
-
-
-
phase
-
string
-
Phase a simple, high-level summary of where the node is in its lifecycle. Can be used as a state machine. Will be one of these values "Pending", "Running" before the node is completed, or "Succeeded", "Skipped", "Failed", "Error", or "Omitted" as a final state.
-
-
-
podIP
-
string
-
PodIP captures the IP of the pod for daemoned steps
-
-
-
progress
-
string
-
Progress to completion
-
-
-
resourcesDuration
-
Map< integer , int64 >
-
ResourcesDuration is indicative, but not accurate, resource duration. This is populated when the nodes completes.
GlobalName exports an output artifact to the global scope, making it available as '{{io.argoproj.workflow.v1alpha1.outputs.artifacts.XXXX}} and in workflow.status.outputs.artifacts
Default is the default value to use for an input parameter if a value was not supplied
-
-
-
description
-
string
-
Description is the parameter description
-
-
-
enum
-
Array< string >
-
Enum holds a list of string values to choose from, for the actual value of the parameter
-
-
-
globalName
-
string
-
GlobalName exports an output parameter to the global scope, making it available as '{{io.argoproj.workflow.v1alpha1.outputs.parameters.XXXX}} and in workflow.status.outputs.parameters
-
-
-
name
-
string
-
Name is the parameter name
-
-
-
value
-
string
-
Value is the literal value to use for the parameter. If specified in the context of an input parameter, the value takes precedence over any passed values
ArtifactLocation describes a location for a single or multiple artifacts. It is used as single artifact in the context of inputs/outputs (e.g. outputs.artifacts.artname). It is also used to describe the location of multiple artifacts such as the archive location of a single workflow step, which the executor will use as a default location to store its files.
RetryStrategy describes how to retry a container nodes in the container set if it fails. Nbr of retries(default 0) and sleep duration between retries(default 0s, instant retry) can be set.
This flag is for DAG logic. The DAG logic has a built-in "fail fast" feature to stop scheduling new steps, as soon as it detects that one of the DAG nodes is failed. Then it waits until all DAG nodes are completed before failing the DAG itself. The FailFast flag default is true, if set to false, it will allow a DAG to run all branches of the DAG to completion (either success or failure), regardless of the failed outcomes of branches in the DAG. More info and example about this feature at https://github.com/argoproj/argo-workflows/issues/1442
-
-
-
target
-
string
-
Target are one or more names of targets to execute in a DAG
Arguments to the entrypoint. The container image's CMD is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. Double $$ are reduced to a single $, which allows for escaping the $(VAR_NAME) syntax: i.e. "$$(VAR_NAME)" will produce the string literal "$(VAR_NAME)". Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell
-
-
-
command
-
Array< string >
-
Entrypoint array. Not executed within a shell. The container image's ENTRYPOINT is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. Double $$ are reduced to a single $, which allows for escaping the $(VAR_NAME) syntax: i.e. "$$(VAR_NAME)" will produce the string literal "$(VAR_NAME)". Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell
List of sources to populate environment variables in the container. The keys defined within a source must be a C_IDENTIFIER. All invalid keys will be reported as an event when the container is starting. When a key exists in multiple sources, the value associated with the last source will take precedence. Values defined by an Env with a duplicate key will take precedence. Cannot be updated.
-
-
-
image
-
string
-
Container image name. More info: https://kubernetes.io/docs/concepts/containers/images This field is optional to allow higher level config management to default or override container images in workload controllers like Deployments and StatefulSets.
-
-
-
imagePullPolicy
-
string
-
Image pull policy. One of Always, Never, IfNotPresent. Defaults to Always if :latest tag is specified, or IfNotPresent otherwise. Cannot be updated. More info: https://kubernetes.io/docs/concepts/containers/images#updating-images
Periodic probe of container liveness. Container will be restarted if the probe fails. Cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes
-
-
-
mirrorVolumeMounts
-
boolean
-
MirrorVolumeMounts will mount the same volumes specified in the main container to the container (including artifacts), at the same mountPaths. This enables dind daemon to partially see the same filesystem as the main container in order to use features such as docker volume binding
-
-
-
name
-
string
-
Name of the container specified as a DNS_LABEL. Each container in a pod must have a unique name (DNS_LABEL). Cannot be updated.
List of ports to expose from the container. Exposing a port here gives the system additional information about the network connections a container uses, but is primarily informational. Not specifying a port here DOES NOT prevent that port from being exposed. Any port which is listening on the default "0.0.0.0" address inside a container will be accessible from the network. Cannot be updated.
Periodic probe of container service readiness. Container will be removed from service endpoints if the probe fails. Cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes
Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
SecurityContext defines the security options the container should be run with. If set, the fields of SecurityContext override the equivalent fields of PodSecurityContext. More info: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/
StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod's lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes
-
-
-
stdin
-
boolean
-
Whether this container should allocate a buffer for stdin in the container runtime. If this is not set, reads from stdin in the container will always result in EOF. Default is false.
-
-
-
stdinOnce
-
boolean
-
Whether the container runtime should close the stdin channel after it has been opened by a single attach. When stdin is true the stdin stream will remain open across multiple attach sessions. If stdinOnce is set to true, stdin is opened on container start, is empty until the first client attaches to stdin, and then remains open and accepts data until the client disconnects, at which time stdin is closed and remains closed until the container is restarted. If this flag is false, a container processes that reads from stdin will never receive an EOF. Default is false
-
-
-
terminationMessagePath
-
string
-
Optional: Path at which the file to which the container's termination message will be written is mounted into the container's filesystem. Message written is intended to be brief final status, such as an assertion failure message. Will be truncated by the node if greater than 4096 bytes. The total message length across all containers will be limited to 12kb. Defaults to /dev/termination-log. Cannot be updated.
-
-
-
terminationMessagePolicy
-
string
-
Indicate how the termination message should be populated. File will use the contents of terminationMessagePath to populate the container status message on both success and failure. FallbackToLogsOnError will use the last chunk of container log output if the termination message file is empty and the container exited with an error. The log output is limited to 2048 bytes or 80 lines, whichever is smaller. Defaults to File. Cannot be updated.
-
-
-
tty
-
boolean
-
Whether this container should allocate a TTY for itself, also requires 'stdin' to be true. Default is false.
Pod volumes to mount into the container's filesystem. Cannot be updated.
-
-
-
workingDir
-
string
-
Container's working directory. If not specified, the container runtime's default will be used, which might be configured in the container image. Cannot be updated.
Action is the action to perform to the resource. Must be one of: get, create, apply, delete, replace, patch
-
-
-
failureCondition
-
string
-
FailureCondition is a label selector expression which describes the conditions of the k8s resource in which the step was considered failed
-
-
-
flags
-
Array< string >
-
Flags is a set of additional options passed to kubectl before submitting a resource I.e. to disable resource validation: flags: [ "--validate=false" # disable resource validation ]
ManifestFrom is the source for a single kubernetes manifest
-
-
-
mergeStrategy
-
string
-
MergeStrategy is the strategy used to merge a patch. It defaults to "strategic" Must be one of: strategic, merge, json
-
-
-
setOwnerReference
-
boolean
-
SetOwnerReference sets the reference to the workflow on the OwnerReference of generated resource.
-
-
-
successCondition
-
string
-
SuccessCondition is a label selector expression which describes the conditions of the k8s resource in which it is acceptable to proceed to the following step
Arguments to the entrypoint. The container image's CMD is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. Double $$ are reduced to a single $, which allows for escaping the $(VAR_NAME) syntax: i.e. "$$(VAR_NAME)" will produce the string literal "$(VAR_NAME)". Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell
-
-
-
command
-
Array< string >
-
Entrypoint array. Not executed within a shell. The container image's ENTRYPOINT is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. Double $$ are reduced to a single $, which allows for escaping the $(VAR_NAME) syntax: i.e. "$$(VAR_NAME)" will produce the string literal "$(VAR_NAME)". Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell
List of sources to populate environment variables in the container. The keys defined within a source must be a C_IDENTIFIER. All invalid keys will be reported as an event when the container is starting. When a key exists in multiple sources, the value associated with the last source will take precedence. Values defined by an Env with a duplicate key will take precedence. Cannot be updated.
-
-
-
image
-
string
-
Container image name. More info: https://kubernetes.io/docs/concepts/containers/images This field is optional to allow higher level config management to default or override container images in workload controllers like Deployments and StatefulSets.
-
-
-
imagePullPolicy
-
string
-
Image pull policy. One of Always, Never, IfNotPresent. Defaults to Always if :latest tag is specified, or IfNotPresent otherwise. Cannot be updated. More info: https://kubernetes.io/docs/concepts/containers/images#updating-images
Periodic probe of container liveness. Container will be restarted if the probe fails. Cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes
-
-
-
name
-
string
-
Name of the container specified as a DNS_LABEL. Each container in a pod must have a unique name (DNS_LABEL). Cannot be updated.
List of ports to expose from the container. Exposing a port here gives the system additional information about the network connections a container uses, but is primarily informational. Not specifying a port here DOES NOT prevent that port from being exposed. Any port which is listening on the default "0.0.0.0" address inside a container will be accessible from the network. Cannot be updated.
Periodic probe of container service readiness. Container will be removed from service endpoints if the probe fails. Cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes
Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
SecurityContext defines the security options the container should be run with. If set, the fields of SecurityContext override the equivalent fields of PodSecurityContext. More info: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/
-
-
-
source
-
string
-
Source contains the source code of the script to execute
StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod's lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes
-
-
-
stdin
-
boolean
-
Whether this container should allocate a buffer for stdin in the container runtime. If this is not set, reads from stdin in the container will always result in EOF. Default is false.
-
-
-
stdinOnce
-
boolean
-
Whether the container runtime should close the stdin channel after it has been opened by a single attach. When stdin is true the stdin stream will remain open across multiple attach sessions. If stdinOnce is set to true, stdin is opened on container start, is empty until the first client attaches to stdin, and then remains open and accepts data until the client disconnects, at which time stdin is closed and remains closed until the container is restarted. If this flag is false, a container processes that reads from stdin will never receive an EOF. Default is false
-
-
-
terminationMessagePath
-
string
-
Optional: Path at which the file to which the container's termination message will be written is mounted into the container's filesystem. Message written is intended to be brief final status, such as an assertion failure message. Will be truncated by the node if greater than 4096 bytes. The total message length across all containers will be limited to 12kb. Defaults to /dev/termination-log. Cannot be updated.
-
-
-
terminationMessagePolicy
-
string
-
Indicate how the termination message should be populated. File will use the contents of terminationMessagePath to populate the container status message on both success and failure. FallbackToLogsOnError will use the last chunk of container log output if the termination message file is empty and the container exited with an error. The log output is limited to 2048 bytes or 80 lines, whichever is smaller. Defaults to File. Cannot be updated.
-
-
-
tty
-
boolean
-
Whether this container should allocate a TTY for itself, also requires 'stdin' to be true. Default is false.
Pod volumes to mount into the container's filesystem. Cannot be updated.
-
-
-
workingDir
-
string
-
Container's working directory. If not specified, the container runtime's default will be used, which might be configured in the container image. Cannot be updated.
Inline is the template. Template must be empty if this is declared (and vice-versa).
-
-
-
name
-
string
-
Name of the step
-
-
-
~~onExit~~
-
~~string~~
-
~~OnExit is a template reference which is invoked at the end of the template, irrespective of the success, failure, or error of the primary template.~~ DEPRECATED: Use Hooks[exit].Template instead.
-
-
-
template
-
string
-
Template is the name of the template to execute as the step
Duration is the seconds to wait before automatically resuming a template. Must be a string. Default unit is seconds. Could also be a Duration, e.g.: "2m", "6h"
ArtifactGC describes how to delete artifacts from completed Workflows - this is embedded into the WorkflowLevelArtifactGC, and also used for individual Artifacts to override that as needed
CreateBucketIfNotPresent tells the driver to attempt to create the S3 bucket for output artifacts, if it doesn't exist. Setting Enabled Encryption will apply either SSE-S3 to the bucket if KmsKeyId is not set or SSE-KMS if it is.
RetryNodeAntiAffinity is a placeholder for future expansion, only empty nodeAntiAffinity is allowed. In order to prevent running steps on the same host, it uses "kubernetes.io/hostname".
Arguments to the entrypoint. The container image's CMD is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. Double $$ are reduced to a single $, which allows for escaping the $(VAR_NAME) syntax: i.e. "$$(VAR_NAME)" will produce the string literal "$(VAR_NAME)". Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell
-
-
-
command
-
Array< string >
-
Entrypoint array. Not executed within a shell. The container image's ENTRYPOINT is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. Double $$ are reduced to a single $, which allows for escaping the $(VAR_NAME) syntax: i.e. "$$(VAR_NAME)" will produce the string literal "$(VAR_NAME)". Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell
List of sources to populate environment variables in the container. The keys defined within a source must be a C_IDENTIFIER. All invalid keys will be reported as an event when the container is starting. When a key exists in multiple sources, the value associated with the last source will take precedence. Values defined by an Env with a duplicate key will take precedence. Cannot be updated.
-
-
-
image
-
string
-
Container image name. More info: https://kubernetes.io/docs/concepts/containers/images This field is optional to allow higher level config management to default or override container images in workload controllers like Deployments and StatefulSets.
-
-
-
imagePullPolicy
-
string
-
Image pull policy. One of Always, Never, IfNotPresent. Defaults to Always if :latest tag is specified, or IfNotPresent otherwise. Cannot be updated. More info: https://kubernetes.io/docs/concepts/containers/images#updating-images
Periodic probe of container liveness. Container will be restarted if the probe fails. Cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes
-
-
-
name
-
string
-
Name of the container specified as a DNS_LABEL. Each container in a pod must have a unique name (DNS_LABEL). Cannot be updated.
List of ports to expose from the container. Exposing a port here gives the system additional information about the network connections a container uses, but is primarily informational. Not specifying a port here DOES NOT prevent that port from being exposed. Any port which is listening on the default "0.0.0.0" address inside a container will be accessible from the network. Cannot be updated.
Periodic probe of container service readiness. Container will be removed from service endpoints if the probe fails. Cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes
Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
SecurityContext defines the security options the container should be run with. If set, the fields of SecurityContext override the equivalent fields of PodSecurityContext. More info: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/
StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod's lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes
-
-
-
stdin
-
boolean
-
Whether this container should allocate a buffer for stdin in the container runtime. If this is not set, reads from stdin in the container will always result in EOF. Default is false.
-
-
-
stdinOnce
-
boolean
-
Whether the container runtime should close the stdin channel after it has been opened by a single attach. When stdin is true the stdin stream will remain open across multiple attach sessions. If stdinOnce is set to true, stdin is opened on container start, is empty until the first client attaches to stdin, and then remains open and accepts data until the client disconnects, at which time stdin is closed and remains closed until the container is restarted. If this flag is false, a container processes that reads from stdin will never receive an EOF. Default is false
-
-
-
terminationMessagePath
-
string
-
Optional: Path at which the file to which the container's termination message will be written is mounted into the container's filesystem. Message written is intended to be brief final status, such as an assertion failure message. Will be truncated by the node if greater than 4096 bytes. The total message length across all containers will be limited to 12kb. Defaults to /dev/termination-log. Cannot be updated.
-
-
-
terminationMessagePolicy
-
string
-
Indicate how the termination message should be populated. File will use the contents of terminationMessagePath to populate the container status message on both success and failure. FallbackToLogsOnError will use the last chunk of container log output if the termination message file is empty and the container exited with an error. The log output is limited to 2048 bytes or 80 lines, whichever is smaller. Defaults to File. Cannot be updated.
-
-
-
tty
-
boolean
-
Whether this container should allocate a TTY for itself, also requires 'stdin' to be true. Default is false.
Pod volumes to mount into the container's filesystem. Cannot be updated.
-
-
-
workingDir
-
string
-
Container's working directory. If not specified, the container runtime's default will be used, which might be configured in the container image. Cannot be updated.
Inline is the template. Template must be empty if this is declared (and vice-versa).
-
-
-
name
-
string
-
Name is the name of the target
-
-
-
~~onExit~~
-
~~string~~
-
~~OnExit is a template reference which is invoked at the end of the template, irrespective of the success, failure, or error of the primary template.~~ DEPRECATED: Use Hooks[exit].Template instead.
ContinueOn defines if a workflow should continue even if a task or step fails/errors. It can be specified if the workflow should continue when the pod errors, fails or both.
CreateBucketIfNotPresent tells the driver to attempt to create the S3 bucket for output artifacts, if it doesn't exist. Setting Enabled Encryption will apply either SSE-S3 to the bucket if KmsKeyId is not set or SSE-KMS if it is.
Holder is a reference to the object which holds the Mutex. Holding Scenario: 1. Current workflow's NodeID which is holding the lock. e.g: ${NodeID} Waiting Scenario: 1. Current workflow or other workflow NodeID which is holding the lock. e.g: ${WorkflowName}/${NodeID}
-
-
-
mutex
-
string
-
Reference for the mutex e.g: ${namespace}/mutex/${mutexName}
NoneStrategy indicates to skip tar process and upload the files or directory tree as independent files. Note that if the artifact is a directory, the artifact driver must support the ability to save/load the directory appropriately.
EnableEncryption tells the driver to encrypt objects if set to true. If kmsKeyId and serverSideCustomerKeySecret are not set, SSE-S3 will be used
-
-
-
kmsEncryptionContext
-
string
-
KmsEncryptionContext is a json blob that contains an encryption context. See https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html#encrypt_context for more information
-
-
-
kmsKeyId
-
string
-
KMSKeyId tells the driver to encrypt the object using the specified KMS Key.
GlobalName exports an output artifact to the global scope, making it available as '{{io.argoproj.workflow.v1alpha1.outputs.artifacts.XXXX}} and in workflow.status.outputs.artifacts
Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata. They are not queryable and should be preserved when modifying objects. More info: http://kubernetes.io/docs/user-guide/annotations
-
-
-
clusterName
-
string
-
The name of the cluster which the object belongs to. This is used to distinguish resources with same name and namespace in different clusters. This field is not set anywhere right now and apiserver is going to ignore it if set in create or update request.
CreationTimestamp is a timestamp representing the server time when this object was created. It is not guaranteed to be set in happens-before order across separate operations. Clients may not set this value. It is represented in RFC3339 form and is in UTC. Populated by the system. Read-only. Null for lists. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata
-
-
-
deletionGracePeriodSeconds
-
integer
-
Number of seconds allowed for this object to gracefully terminate before it will be removed from the system. Only set when deletionTimestamp is also set. May only be shortened. Read-only.
DeletionTimestamp is RFC 3339 date and time at which this resource will be deleted. This field is set by the server when a graceful deletion is requested by the user, and is not directly settable by a client. The resource is expected to be deleted (no longer visible from resource lists, and not reachable by name) after the time in this field, once the finalizers list is empty. As long as the finalizers list contains items, deletion is blocked. Once the deletionTimestamp is set, this value may not be unset or be set further into the future, although it may be shortened or the resource may be deleted prior to this time. For example, a user may request that a pod is deleted in 30 seconds. The Kubelet will react by sending a graceful termination signal to the containers in the pod. After that 30 seconds, the Kubelet will send a hard termination signal (SIGKILL) to the container and after cleanup, remove the pod from the API. In the presence of network partitions, this object may still exist after this timestamp, until an administrator or automated process can determine the resource is fully terminated. If not set, graceful deletion of the object has not been requested. Populated by the system when a graceful deletion is requested. Read-only. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata
-
-
-
finalizers
-
Array< string >
-
Must be empty before the object is deleted from the registry. Each entry is an identifier for the responsible component that will remove the entry from the list. If the deletionTimestamp of the object is non-nil, entries in this list can only be removed. Finalizers may be processed and removed in any order. Order is NOT enforced because it introduces significant risk of stuck finalizers. finalizers is a shared field, any actor with permission can reorder it. If the finalizer list is processed in order, then this can lead to a situation in which the component responsible for the first finalizer in the list is waiting for a signal (field value, external system, or other) produced by a component responsible for a finalizer later in the list, resulting in a deadlock. Without enforced ordering finalizers are free to order amongst themselves and are not vulnerable to ordering changes in the list.
-
-
-
generateName
-
string
-
GenerateName is an optional prefix, used by the server, to generate a unique name ONLY IF the Name field has not been provided. If this field is used, the name returned to the client will be different than the name passed. This value will also be combined with a unique suffix. The provided value has the same validation rules as the Name field, and may be truncated by the length of the suffix required to make the value unique on the server. If this field is specified and the generated name exists, the server will NOT return a 409 - instead, it will either return 201 Created or 500 with Reason ServerTimeout indicating a unique name could not be found in the time allotted, and the client should retry (optionally after the time indicated in the Retry-After header). Applied only if Name is not specified. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#idempotency
-
-
-
generation
-
integer
-
A sequence number representing a specific generation of the desired state. Populated by the system. Read-only.
-
-
-
labels
-
Map< string , string >
-
Map of string keys and values that can be used to organize and categorize (scope and select) objects. May match selectors of replication controllers and services. More info: http://kubernetes.io/docs/user-guide/labels
ManagedFields maps workflow-id and version to the set of fields that are managed by that workflow. This is mostly for internal housekeeping, and users typically shouldn't need to set or understand this field. A workflow can be the user's name, a controller's name, or the name of a specific apply path like "ci-cd". The set of fields is always in the version that the workflow used when modifying the object.
-
-
-
name
-
string
-
Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/identifiers#names
-
-
-
namespace
-
string
-
Namespace defines the space within which each name must be unique. An empty namespace is equivalent to the "default" namespace, but "default" is the canonical representation. Not all objects are required to be scoped to a namespace - the value of this field for those objects will be empty. Must be a DNS_LABEL. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/namespaces
List of objects depended by this object. If ALL objects in the list have been deleted, this object will be garbage collected. If this object is managed by a controller, then an entry in this list will point to this controller, with the controller field set to true. There cannot be more than one managing controller.
-
-
-
resourceVersion
-
string
-
An opaque value that represents the internal version of this object that can be used by clients to determine when objects have changed. May be used for optimistic concurrency, change detection, and the watch operation on a resource or set of resources. Clients must treat these values as opaque and passed unmodified back to the server. They may only be valid for a particular resource or set of resources. Populated by the system. Read-only. Value must be treated as opaque by clients and . More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency
-
-
-
~~selfLink~~
-
~~string~~
-
~~SelfLink is a URL representing this object. Populated by the system. Read-only.~~ DEPRECATED Kubernetes will stop propagating this field in 1.20 release and the field is planned to be removed in 1.21 release.
-
-
-
uid
-
string
-
UID is the unique in time and space value for this object. It is typically generated by the server on successful creation of a resource and is not allowed to change on PUT operations. Populated by the system. Read-only. More info: http://kubernetes.io/docs/user-guide/identifiers#uids
A list of DNS name server IP addresses. This will be appended to the base nameservers generated from DNSPolicy. Duplicated nameservers will be removed.
A list of DNS resolver options. This will be merged with the base options generated from DNSPolicy. Duplicated entries will be removed. Resolution options given in Options will override those that appear in the base DNSPolicy.
-
-
-
searches
-
Array< string >
-
A list of DNS search domains for host-name lookup. This will be appended to the base search paths generated from DNSPolicy. Duplicated search paths will be removed.
An eviction is allowed if at most "maxUnavailable" pods selected by "selector" are unavailable after the eviction, i.e. even in absence of the evicted pod. For example, one can prevent all voluntary evictions by specifying 0. This is a mutually exclusive setting with "minAvailable".
An eviction is allowed if at least "minAvailable" pods selected by "selector" will still be available after the eviction, i.e. even in the absence of the evicted pod. So for example you can prevent all voluntary evictions by specifying "100%".
Label query over pods whose evictions are managed by the disruption budget. A null selector will match no pods, while an empty ({}) selector will select all pods within the namespace.
PodSecurityContext holds pod-level security attributes and common container settings. Some fields are also present in container.securityContext. Field values of container.securityContext take precedence over field values of PodSecurityContext.
A special supplemental group that applies to all containers in a pod. Some volume types allow the Kubelet to change the ownership of that volume to be owned by the pod: 1. The owning GID will be the FSGroup 2. The setgid bit is set (new files created in the volume will be owned by FSGroup) 3. The permission bits are OR'd with rw-rw---- If unset, the Kubelet will not modify the ownership and permissions of any volume. Note that this field cannot be set when spec.os.name is windows.
-
-
-
fsGroupChangePolicy
-
string
-
fsGroupChangePolicy defines behavior of changing ownership and permission of the volume before being exposed inside Pod. This field will only apply to volume types which support fsGroup based ownership(and permissions). It will have no effect on ephemeral volume types such as: secret, configmaps and emptydir. Valid values are "OnRootMismatch" and "Always". If not specified, "Always" is used. Note that this field cannot be set when spec.os.name is windows.
-
-
-
runAsGroup
-
integer
-
The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container. Note that this field cannot be set when spec.os.name is windows.
-
-
-
runAsNonRoot
-
boolean
-
Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.
-
-
-
runAsUser
-
integer
-
The UID to run the entrypoint of the container process. Defaults to user specified in image metadata if unspecified. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container. Note that this field cannot be set when spec.os.name is windows.
The SELinux context to be applied to all containers. If unspecified, the container runtime will allocate a random SELinux context for each container. May also be set in SecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence for that container. Note that this field cannot be set when spec.os.name is windows.
The seccomp options to use by the containers in this pod. Note that this field cannot be set when spec.os.name is windows.
-
-
-
supplementalGroups
-
Array< integer >
-
A list of groups applied to the first process run in each container, in addition to the container's primary GID. If unspecified, no groups will be added to any container. Note that this field cannot be set when spec.os.name is windows.
Sysctls hold a list of namespaced sysctls used for the pod. Pods with unsupported sysctls (by the container runtime) might fail to launch. Note that this field cannot be set when spec.os.name is windows.
The Windows specific settings applied to all containers. If unspecified, the options within a container's SecurityContext will be used. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence. Note that this field cannot be set when spec.os.name is linux.
Effect indicates the taint effect to match. Empty means match all taint effects. When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute. Possible enum values: - "NoExecute" Evict any already-running pods that do not tolerate the taint. Currently enforced by NodeController. - "NoSchedule" Do not allow new pods to schedule onto the node unless they tolerate the taint, but allow all pods submitted to Kubelet without going through the scheduler to start, and allow all already-running pods to continue running. Enforced by the scheduler. - "PreferNoSchedule" Like TaintEffectNoSchedule, but the scheduler tries not to schedule new pods onto the node, rather than prohibiting new pods from scheduling onto the node entirely. Enforced by the scheduler.
-
-
-
key
-
string
-
Key is the taint key that the toleration applies to. Empty means match all taint keys. If the key is empty, operator must be Exists; this combination means to match all values and all keys.
-
-
-
operator
-
string
-
Operator represents a key's relationship to the value. Valid operators are Exists and Equal. Defaults to Equal. Exists is equivalent to wildcard for value, so that a pod can tolerate all taints of a particular category. Possible enum values: - "Equal" - "Exists"
-
-
-
tolerationSeconds
-
integer
-
TolerationSeconds represents the period of time the toleration (which must be of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default, it is not set, which means tolerate the taint forever (do not evict). Zero and negative values will be treated as 0 (evict immediately) by the system.
-
-
-
value
-
string
-
Value is the taint value the toleration matches to. If the operator is Exists, the value should be empty, otherwise just a regular string.
APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
-
-
-
kind
-
string
-
Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
Spec defines the desired characteristics of a volume requested by a pod author. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims
Status represents the current information/status of a persistent volume claim. Read-only. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims
AWSElasticBlockStore represents an AWS Disk resource that is attached to a kubelet's host machine and then exposed to the pod. More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore
Ephemeral represents a volume that is handled by a cluster storage driver. The volume's lifecycle is tied to the pod that defines it - it will be created before the pod starts, and deleted when the pod is removed. Use this if: a) the volume is only needed while the pod runs, b) features of normal volumes like restoring from snapshot or capacity tracking are needed, c) the storage driver is specified through a storage class, and d) the storage driver supports dynamic volume provisioning through a PersistentVolumeClaim (see EphemeralVolumeSource for more information on the connection between this volume type and PersistentVolumeClaim). Use PersistentVolumeClaim or one of the vendor-specific APIs for volumes that persist for longer than the lifecycle of an individual pod. Use CSI for light-weight local ephemeral volumes if the CSI driver is meant to be used that way - see the documentation of the driver for more information. A pod can use both types of ephemeral volumes and persistent volumes at the same time.
GCEPersistentDisk represents a GCE Disk resource that is attached to a kubelet's host machine and then exposed to the pod. More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk
~~GitRepo represents a git repository at a particular revision.~~ DEPRECATED: GitRepo is deprecated. To provision a container with a git repo, mount an EmptyDir into an InitContainer that clones the repo using git, then mount the EmptyDir into the Pod's container.
HostPath represents a pre-existing file or directory on the host machine that is directly exposed to the container. This is generally used for system agents or other privileged things that are allowed to see the host machine. Most containers will NOT need this. More info: https://kubernetes.io/docs/concepts/storage/volumes#hostpath
ISCSI represents an ISCSI Disk resource that is attached to a kubelet's host machine and then exposed to the pod. More info: https://examples.k8s.io/volumes/iscsi/README.md
-
-
-
name
-
string
-
Volume's name. Must be a DNS_LABEL and unique within the pod. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
PersistentVolumeClaimVolumeSource represents a reference to a PersistentVolumeClaim in the same namespace. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims
Time is a wrapper around time.Time which supports correct marshaling to YAML and JSON. Wrappers are provided for many of the factory methods that the time package offers.
If referring to a piece of an object instead of an entire object, this string should contain a valid JSON/Go field access statement, such as desiredState.manifest.containers[2]. For example, if the object reference is to a container within a pod, this would take on a value like: "spec.containers{name}" (where "name" refers to the name of the container that triggered the event) or if no container name is specified "spec.containers[2]" (container with index 2 in this pod). This syntax is chosen only to have some well-defined way of referencing a part of an object.
-
-
-
kind
-
string
-
Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
-
-
-
name
-
string
-
Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
-
-
-
namespace
-
string
-
Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/
-
-
-
resourceVersion
-
string
-
Specific resourceVersion to which this reference is made, if any. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency
-
-
-
uid
-
string
-
UID of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#uids
Duration is a wrapper around time.Duration which supports correct marshaling to YAML and JSON. In particular, it marshals into strings, which can be used as map keys in json.
A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all objects. A null label selector matches no objects.
matchExpressions is a list of label selector requirements. The requirements are ANDed.
-
-
-
matchLabels
-
Map< string , string >
-
matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels map is equivalent to an element of matchExpressions, whose key field is "key", the operator is "In", and the values array contains only "value". The requirements are ANDed.
Arguments to the entrypoint. The docker image's CMD is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. Double $$ are reduced to a single $, which allows for escaping the $(VAR_NAME) syntax: i.e. "$$(VAR_NAME)" will produce the string literal "$(VAR_NAME)". Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell
-
-
-
command
-
Array< string >
-
Entrypoint array. Not executed within a shell. The docker image's ENTRYPOINT is used if this is not provided. Variable references $(VAR_NAME) are expanded using the container's environment. If a variable cannot be resolved, the reference in the input string will be unchanged. Double $$ are reduced to a single $, which allows for escaping the $(VAR_NAME) syntax: i.e. "$$(VAR_NAME)" will produce the string literal "$(VAR_NAME)". Escaped references will never be expanded, regardless of whether the variable exists or not. Cannot be updated. More info: https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#running-a-command-in-a-shell
List of sources to populate environment variables in the container. The keys defined within a source must be a C_IDENTIFIER. All invalid keys will be reported as an event when the container is starting. When a key exists in multiple sources, the value associated with the last source will take precedence. Values defined by an Env with a duplicate key will take precedence. Cannot be updated.
-
-
-
image
-
string
-
Docker image name. More info: https://kubernetes.io/docs/concepts/containers/images This field is optional to allow higher level config management to default or override container images in workload controllers like Deployments and StatefulSets.
-
-
-
imagePullPolicy
-
string
-
Image pull policy. One of Always, Never, IfNotPresent. Defaults to Always if :latest tag is specified, or IfNotPresent otherwise. Cannot be updated. More info: https://kubernetes.io/docs/concepts/containers/images#updating-images Possible enum values: - "Always" means that kubelet always attempts to pull the latest image. Container will fail If the pull fails. - "IfNotPresent" means that kubelet pulls if the image isn't present on disk. Container will fail if the image isn't present and the pull fails. - "Never" means that kubelet never pulls an image, but only uses a local image. Container will fail if the image isn't present
Periodic probe of container liveness. Container will be restarted if the probe fails. Cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes
-
-
-
name
-
string
-
Name of the container specified as a DNS_LABEL. Each container in a pod must have a unique name (DNS_LABEL). Cannot be updated.
List of ports to expose from the container. Exposing a port here gives the system additional information about the network connections a container uses, but is primarily informational. Not specifying a port here DOES NOT prevent that port from being exposed. Any port which is listening on the default "0.0.0.0" address inside a container will be accessible from the network. Cannot be updated.
Periodic probe of container service readiness. Container will be removed from service endpoints if the probe fails. Cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes
Compute Resources required by this container. Cannot be updated. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
SecurityContext defines the security options the container should be run with. If set, the fields of SecurityContext override the equivalent fields of PodSecurityContext. More info: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/
StartupProbe indicates that the Pod has successfully initialized. If specified, no other probes are executed until this completes successfully. If this probe fails, the Pod will be restarted, just as if the livenessProbe failed. This can be used to provide different probe parameters at the beginning of a Pod's lifecycle, when it might take a long time to load data or warm a cache, than during steady-state operation. This cannot be updated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes
-
-
-
stdin
-
boolean
-
Whether this container should allocate a buffer for stdin in the container runtime. If this is not set, reads from stdin in the container will always result in EOF. Default is false.
-
-
-
stdinOnce
-
boolean
-
Whether the container runtime should close the stdin channel after it has been opened by a single attach. When stdin is true the stdin stream will remain open across multiple attach sessions. If stdinOnce is set to true, stdin is opened on container start, is empty until the first client attaches to stdin, and then remains open and accepts data until the client disconnects, at which time stdin is closed and remains closed until the container is restarted. If this flag is false, a container processes that reads from stdin will never receive an EOF. Default is false
-
-
-
terminationMessagePath
-
string
-
Optional: Path at which the file to which the container's termination message will be written is mounted into the container's filesystem. Message written is intended to be brief final status, such as an assertion failure message. Will be truncated by the node if greater than 4096 bytes. The total message length across all containers will be limited to 12kb. Defaults to /dev/termination-log. Cannot be updated.
-
-
-
terminationMessagePolicy
-
string
-
Indicate how the termination message should be populated. File will use the contents of terminationMessagePath to populate the container status message on both success and failure. FallbackToLogsOnError will use the last chunk of container log output if the termination message file is empty and the container exited with an error. The log output is limited to 2048 bytes or 80 lines, whichever is smaller. Defaults to File. Cannot be updated. Possible enum values: - "FallbackToLogsOnError" will read the most recent contents of the container logs for the container status message when the container exits with an error and the terminationMessagePath has no contents. - "File" is the default behavior and will set the container status message to the contents of the container's terminationMessagePath when the container exits.
-
-
-
tty
-
boolean
-
Whether this container should allocate a TTY for itself, also requires 'stdin' to be true. Default is false.
Pod volumes to mount into the container's filesystem. Cannot be updated.
-
-
-
workingDir
-
string
-
Container's working directory. If not specified, the container runtime's default will be used, which might be configured in the container image. Cannot be updated.
Path within the container at which the volume should be mounted. Must not contain ':'.
-
-
-
mountPropagation
-
string
-
mountPropagation determines how mounts are propagated from the host to container and the other way around. When not set, MountPropagationNone is used. This field is beta in 1.10.
-
-
-
name
-
string
-
This must match the Name of a Volume.
-
-
-
readOnly
-
boolean
-
Mounted read-only if true, read-write otherwise (false or unspecified). Defaults to false.
-
-
-
subPath
-
string
-
Path within the volume from which the container's volume should be mounted. Defaults to "" (volume's root).
-
-
-
subPathExpr
-
string
-
Expanded path within the volume from which the container's volume should be mounted. Behaves similarly to SubPath but environment variable references $(VAR_NAME) are expanded using the container's environment. Defaults to "" (volume's root). SubPathExpr and SubPath are mutually exclusive.
Name of the environment variable. Must be a C_IDENTIFIER.
-
-
-
value
-
string
-
Variable references $(VAR_NAME) are expanded using the previously defined environment variables in the container and any service environment variables. If a variable cannot be resolved, the reference in the input string will be unchanged. Double $$ are reduced to a single $, which allows for escaping the $(VAR_NAME) syntax: i.e. "$$(VAR_NAME)" will produce the string literal "$(VAR_NAME)". Escaped references will never be expanded, regardless of whether the variable exists or not. Defaults to "".
Lifecycle describes actions that the management system should take in response to container lifecycle events. For the PostStart and PreStop lifecycle handlers, management of the container blocks until the action is complete, unless the container process fails, in which case the handler is aborted.
PostStart is called immediately after a container is created. If the handler fails, the container is terminated and restarted according to its restart policy. Other management of the container blocks until the hook completes. More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks
PreStop is called immediately before a container is terminated due to an API request or management event such as liveness/startup probe failure, preemption, resource contention, etc. The handler is not called if the container crashes or exits. The Pod's termination grace period countdown begins before the PreStop hook is executed. Regardless of the outcome of the handler, the container will eventually terminate within the Pod's termination grace period (unless delayed by finalizers). Other management of the container blocks until the hook completes or until the termination grace period is reached. More info: https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks
Number of seconds after the container has started before liveness probes are initiated. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes
-
-
-
periodSeconds
-
integer
-
How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.
-
-
-
successThreshold
-
integer
-
Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness and startup. Minimum value is 1.
TCPSocket specifies an action involving a TCP port.
-
-
-
terminationGracePeriodSeconds
-
integer
-
Optional duration in seconds the pod needs to terminate gracefully upon probe failure. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with a kill signal. Set this value longer than the expected cleanup time for your process. If this value is nil, the pod's terminationGracePeriodSeconds will be used. Otherwise, this value overrides the value provided by the pod spec. Value must be non-negative integer. The value zero indicates stop immediately via the kill signal (no opportunity to shut down). This is a beta field and requires enabling ProbeTerminationGracePeriod feature gate. Minimum value is 1. spec.terminationGracePeriodSeconds is used if unset.
-
-
-
timeoutSeconds
-
integer
-
Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1. More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#container-probes
Number of port to expose on the pod's IP address. This must be a valid port number, 0 < x < 65536.
-
-
-
hostIP
-
string
-
What host IP to bind the external port to.
-
-
-
hostPort
-
integer
-
Number of port to expose on the host. If specified, this must be a valid port number, 0 < x < 65536. If HostNetwork is specified, this must match ContainerPort. Most containers do not need this.
-
-
-
name
-
string
-
If specified, this must be an IANA_SVC_NAME and unique within the pod. Each named port in a pod must have a unique name. Name for the port that can be referred to by services.
-
-
-
protocol
-
string
-
Protocol for port. Must be UDP, TCP, or SCTP. Defaults to "TCP". Possible enum values: - "SCTP" is the SCTP protocol. - "TCP" is the TCP protocol. - "UDP" is the UDP protocol.
Limits describes the maximum amount of compute resources allowed. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
Requests describes the minimum amount of compute resources required. If Requests is omitted for a container, it defaults to Limits if that is explicitly specified, otherwise to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
SecurityContext holds security configuration that will be applied to a container. Some fields are present in both SecurityContext and PodSecurityContext. When both are set, the values in SecurityContext take precedence.
AllowPrivilegeEscalation controls whether a process can gain more privileges than its parent process. This bool directly controls if the no_new_privs flag will be set on the container process. AllowPrivilegeEscalation is true always when the container is: 1) run as Privileged 2) has CAP_SYS_ADMIN Note that this field cannot be set when spec.os.name is windows.
The capabilities to add/drop when running containers. Defaults to the default set of capabilities granted by the container runtime. Note that this field cannot be set when spec.os.name is windows.
-
-
-
privileged
-
boolean
-
Run container in privileged mode. Processes in privileged containers are essentially equivalent to root on the host. Defaults to false. Note that this field cannot be set when spec.os.name is windows.
-
-
-
procMount
-
string
-
procMount denotes the type of proc mount to use for the containers. The default is DefaultProcMount which uses the container runtime defaults for readonly paths and masked paths. This requires the ProcMountType feature flag to be enabled. Note that this field cannot be set when spec.os.name is windows.
-
-
-
readOnlyRootFilesystem
-
boolean
-
Whether this container has a read-only root filesystem. Default is false. Note that this field cannot be set when spec.os.name is windows.
-
-
-
runAsGroup
-
integer
-
The GID to run the entrypoint of the container process. Uses runtime default if unset. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence. Note that this field cannot be set when spec.os.name is windows.
-
-
-
runAsNonRoot
-
boolean
-
Indicates that the container must run as a non-root user. If true, the Kubelet will validate the image at runtime to ensure that it does not run as UID 0 (root) and fail to start the container if it does. If unset or false, no such validation will be performed. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.
-
-
-
runAsUser
-
integer
-
The UID to run the entrypoint of the container process. Defaults to user specified in image metadata if unspecified. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence. Note that this field cannot be set when spec.os.name is windows.
The SELinux context to be applied to the container. If unspecified, the container runtime will allocate a random SELinux context for each container. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence. Note that this field cannot be set when spec.os.name is windows.
The seccomp options to use by this container. If seccomp options are provided at both the pod & container level, the container options override the pod options. Note that this field cannot be set when spec.os.name is windows.
The Windows specific settings applied to all containers. If unspecified, the options from the PodSecurityContext will be used. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence. Note that this field cannot be set when spec.os.name is linux.
APIVersion defines the version of this resource that this field set applies to. The format is "group/version" just like the top-level APIVersion field. It is necessary to track the version of a field set because it cannot be automatically converted.
-
-
-
fieldsType
-
string
-
FieldsType is the discriminator for the different fields format and version. There is currently only one possible value: "FieldsV1"
FieldsV1 holds the first JSON version format as described in the "FieldsV1" type.
-
-
-
manager
-
string
-
Manager is an identifier of the workflow managing these fields.
-
-
-
operation
-
string
-
Operation is the type of operation which lead to this ManagedFieldsEntry being created. The only valid values for this field are 'Apply' and 'Update'.
-
-
-
subresource
-
string
-
Subresource is the name of the subresource used to update that object, or empty string if the object was updated through the main resource. The value of this field is used to distinguish between managers, even if they share the same name. For example, a status update will be distinct from a regular update using the same manager name. Note that the APIVersion field is not related to the Subresource field and it always corresponds to the version of the main resource.
OwnerReference contains enough information to let you identify an owning object. An owning object must be in the same namespace as the dependent, or be cluster-scoped, so there is no namespace field.
If true, AND if the owner has the "foregroundDeletion" finalizer, then the owner cannot be deleted from the key-value store until this reference is removed. Defaults to false. To set this field, a user needs "delete" permission of the owner, otherwise 422 (Unprocessable Entity) will be returned.
-
-
-
controller
-
boolean
-
If true, this reference points to the managing controller.
-
-
-
kind
-
string
-
Kind of the referent. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
-
-
-
name
-
string
-
Name of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#names
-
-
-
uid
-
string
-
UID of the referent. More info: http://kubernetes.io/docs/user-guide/identifiers#uids
The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding "weight" to the sum if the node matches the corresponding matchExpressions; the node(s) with the highest sum are the most preferred.
If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to an update), the system may or may not try to eventually evict the pod from its node.
The scheduler will prefer to schedule pods to nodes that satisfy the affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding "weight" to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.
If the affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.
The scheduler will prefer to schedule pods to nodes that satisfy the anti-affinity expressions specified by this field, but it may choose a node that violates one or more of the expressions. The node that is most preferred is the one with the greatest sum of weights, i.e. for each node that meets all of the scheduling requirements (resource request, requiredDuringScheduling anti-affinity expressions, etc.), compute a sum by iterating through the elements of this field and adding "weight" to the sum if the node has pods which matches the corresponding podAffinityTerm; the node(s) with the highest sum are the most preferred.
If the anti-affinity requirements specified by this field are not met at scheduling time, the pod will not be scheduled onto the node. If the anti-affinity requirements specified by this field cease to be met at some point during pod execution (e.g. due to a pod label update), the system may or may not try to eventually evict the pod from its node. When there are multiple elements, the lists of nodes corresponding to each podAffinityTerm are intersected, i.e. all terms must be satisfied.
localhostProfile indicates a profile defined in a file on the node should be used. The profile must be preconfigured on the node to work. Must be a descending path, relative to the kubelet's configured seccomp profile location. Must only be set if type is "Localhost".
-
-
-
type
-
string
-
type indicates which kind of seccomp profile will be applied. Valid options are: Localhost - a profile defined in a file on the node should be used. RuntimeDefault - the container runtime default profile should be used. Unconfined - no profile should be applied. Possible enum values: - "Localhost" indicates a profile defined in a file on the node should be used. The file's location relative to /seccomp. - "RuntimeDefault" represents the default container runtime seccomp profile. - "Unconfined" indicates no seccomp profile is applied (A.K.A. unconfined).
GMSACredentialSpec is where the GMSA admission webhook (https://github.com/kubernetes-sigs/windows-gmsa) inlines the contents of the GMSA credential spec named by the GMSACredentialSpecName field.
-
-
-
gmsaCredentialSpecName
-
string
-
GMSACredentialSpecName is the name of the GMSA credential spec to use.
-
-
-
hostProcess
-
boolean
-
HostProcess determines if a container should be run as a 'Host Process' container. This field is alpha-level and will only be honored by components that enable the WindowsHostProcessContainers feature flag. Setting this field without the feature flag will result in errors when validating the Pod. All of a Pod's containers must have the same effective HostProcess value (it is not allowed to have a mix of HostProcess containers and non-HostProcess containers). In addition, if HostProcess is true then HostNetwork must also be set to true.
-
-
-
runAsUserName
-
string
-
The UserName in Windows to run the entrypoint of the container process. Defaults to the user specified in image metadata if unspecified. May also be set in PodSecurityContext. If set in both SecurityContext and PodSecurityContext, the value specified in SecurityContext takes precedence.
AccessModes contains the desired access modes the volume should have. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1
This field can be used to specify either: * An existing VolumeSnapshot object (snapshot.storage.k8s.io/VolumeSnapshot) * An existing PVC (PersistentVolumeClaim) If the provisioner or an external controller can support the specified data source, it will create a new volume based on the contents of the specified data source. If the AnyVolumeDataSource feature gate is enabled, this field will always have the same contents as the DataSourceRef field.
Specifies the object from which to populate the volume with data, if a non-empty volume is desired. This may be any local object from a non-empty API group (non core object) or a PersistentVolumeClaim object. When this field is specified, volume binding will only succeed if the type of the specified object matches some installed volume populator or dynamic provisioner. This field will replace the functionality of the DataSource field and as such if both fields are non-empty, they must have the same value. For backwards compatibility, both fields (DataSource and DataSourceRef) will be set to the same value automatically if one of them is empty and the other is non-empty. There are two important differences between DataSource and DataSourceRef: * While DataSource only allows two specific types of objects, DataSourceRef allows any non-core object, as well as PersistentVolumeClaim objects. * While DataSource ignores disallowed values (dropping them), DataSourceRef preserves all values, and generates an error if a disallowed value is specified. (Alpha) Using this field requires the AnyVolumeDataSource feature gate to be enabled.
Resources represents the minimum resources the volume should have. If RecoverVolumeExpansionFailure feature is enabled users are allowed to specify resource requirements that are lower than previous value but must still be higher than capacity recorded in the status field of the claim. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#resources
AccessModes contains the actual access modes the volume backing the PVC has. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#access-modes-1
The storage resource within AllocatedResources tracks the capacity allocated to a PVC. It may be larger than the actual capacity when a volume expansion operation is requested. For storage quota, the larger value from allocatedResources and PVC.spec.resources is used. If allocatedResources is not set, PVC.spec.resources alone is used for quota calculation. If a volume expansion capacity request is lowered, allocatedResources is only lowered if there are no expansion operations in progress and if the actual volume capacity is equal or lower than the requested capacity. This is an alpha field and requires enabling RecoverVolumeExpansionFailure feature.
Current Condition of persistent volume claim. If underlying persistent volume is being resized then the Condition will be set to 'ResizeStarted'.
-
-
-
phase
-
string
-
Phase represents the current phase of PersistentVolumeClaim. Possible enum values: - "Bound" used for PersistentVolumeClaims that are bound - "Lost" used for PersistentVolumeClaims that lost their underlying PersistentVolume. The claim was bound to a PersistentVolume and this volume does not exist any longer and all data on it was lost. - "Pending" used for PersistentVolumeClaims that are not yet bound
-
-
-
resizeStatus
-
string
-
ResizeStatus stores status of resize operation. ResizeStatus is not set by default but when expansion is complete resizeStatus is set to empty string by resize controller or kubelet. This is an alpha field and requires enabling RecoverVolumeExpansionFailure feature.
Represents a Persistent Disk resource in AWS. An AWS EBS disk must exist before mounting to a container. The disk must also be in the same AWS zone as the kubelet. An AWS EBS disk can only be mounted as read/write once. AWS EBS volumes support ownership management and SELinux relabeling.
Filesystem type of the volume that you want to mount. Tip: Ensure that the filesystem type is supported by the host operating system. Examples: "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" if unspecified. More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore
-
-
-
partition
-
integer
-
The partition in the volume that you want to mount. If omitted, the default is to mount by volume name. Examples: For volume /dev/sda1, you specify the partition as "1". Similarly, the volume partition for /dev/sda is "0" (or you can leave the property empty).
-
-
-
readOnly
-
boolean
-
Specify "true" to force and set the ReadOnly property in VolumeMounts to "true". If omitted, the default is "false". More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore
-
-
-
volumeID
-
string
-
Unique ID of the persistent disk resource in AWS (Amazon EBS volume). More info: https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore
Filesystem type to mount. Must be a filesystem type supported by the host operating system. Ex. "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" if unspecified.
-
-
-
kind
-
string
-
Expected values Shared: multiple blob disks per storage account Dedicated: single blob disk per storage account Managed: azure managed data disk (only in managed availability set). defaults to shared
-
-
-
readOnly
-
boolean
-
Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts.
Required: Monitors is a collection of Ceph monitors More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it
-
-
-
path
-
string
-
Optional: Used as the mounted root, rather than the full Ceph tree, default is /
-
-
-
readOnly
-
boolean
-
Optional: Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts. More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it
-
-
-
secretFile
-
string
-
Optional: SecretFile is the path to key ring for User, default is /etc/ceph/user.secret More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it
Optional: SecretRef is reference to the authentication secret for User, default is empty. More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it
-
-
-
user
-
string
-
Optional: User is the rados user name, default is admin More info: https://examples.k8s.io/volumes/cephfs/README.md#how-to-use-it
Represents a cinder volume resource in Openstack. A Cinder volume must exist before mounting to a container. The volume must also be in the same region as the kubelet. Cinder volumes support ownership management and SELinux relabeling.
Filesystem type to mount. Must be a filesystem type supported by the host operating system. Examples: "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" if unspecified. More info: https://examples.k8s.io/mysql-cinder-pd/README.md
-
-
-
readOnly
-
boolean
-
Optional: Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts. More info: https://examples.k8s.io/mysql-cinder-pd/README.md
Adapts a ConfigMap into a volume. The contents of the target ConfigMap's Data field will be presented in a volume as files using the keys in the Data field as the file names, unless the items element is populated with specific mappings of keys to paths. ConfigMap volumes support ownership management and SELinux relabeling.
Optional: mode bits used to set permissions on created files by default. Must be an octal value between 0000 and 0777 or a decimal value between 0 and 511. YAML accepts both octal and decimal values, JSON requires decimal values for mode bits. Defaults to 0644. Directories within the path are not affected by this setting. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set.
If unspecified, each key-value pair in the Data field of the referenced ConfigMap will be projected into the volume as a file whose name is the key and content is the value. If specified, the listed keys will be projected into the specified paths, and unlisted keys will not be present. If a key is specified which is not present in the ConfigMap, the volume setup will error unless it is marked optional. Paths must be relative and may not contain the '..' path or start with '..'.
-
-
-
name
-
string
-
Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
-
-
-
optional
-
boolean
-
Specify whether the ConfigMap or its keys must be defined
Driver is the name of the CSI driver that handles this volume. Consult with your admin for the correct name as registered in the cluster.
-
-
-
fsType
-
string
-
Filesystem type to mount. Ex. "ext4", "xfs", "ntfs". If not provided, the empty value is passed to the associated CSI driver which will determine the default filesystem to apply.
NodePublishSecretRef is a reference to the secret object containing sensitive information to pass to the CSI driver to complete the CSI NodePublishVolume and NodeUnpublishVolume calls. This field is optional, and may be empty if no secret is required. If the secret object contains more than one secret, all secret references are passed.
-
-
-
readOnly
-
boolean
-
Specifies a read-only configuration for the volume. Defaults to false (read/write).
-
-
-
volumeAttributes
-
Map< string , string >
-
VolumeAttributes stores driver-specific properties that are passed to the CSI driver. Consult your driver's documentation for supported values.
Optional: mode bits to use on created files by default. Must be a Optional: mode bits used to set permissions on created files by default. Must be an octal value between 0000 and 0777 or a decimal value between 0 and 511. YAML accepts both octal and decimal values, JSON requires decimal values for mode bits. Defaults to 0644. Directories within the path are not affected by this setting. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set.
What type of storage medium should back this directory. The default is "" which means to use the node's default medium. Must be an empty string (default) or Memory. More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir
Total amount of local storage required for this EmptyDir volume. The size limit is also applicable for memory medium. The maximum usage on memory medium EmptyDir would be the minimum value between the SizeLimit specified here and the sum of memory limits of all containers in a pod. The default is nil which means that the limit is undefined. More info: http://kubernetes.io/docs/user-guide/volumes#emptydir
Will be used to create a stand-alone PVC to provision the volume. The pod in which this EphemeralVolumeSource is embedded will be the owner of the PVC, i.e. the PVC will be deleted together with the pod. The name of the PVC will be <pod name>-<volume name> where <volume name> is the name from the PodSpec.Volumes array entry. Pod validation will reject the pod if the concatenated name is not valid for a PVC (for example, too long). An existing PVC with that name that is not owned by the pod will not be used for the pod to avoid using an unrelated volume by mistake. Starting the pod is then blocked until the unrelated PVC is removed. If such a pre-created PVC is meant to be used by the pod, the PVC has to updated with an owner reference to the pod once the pod exists. Normally this should not be necessary, but it may be useful when manually reconstructing a broken cluster. This field is read-only and no changes will be made by Kubernetes to the PVC after it has been created. Required, must not be nil.
Represents a Fibre Channel volume. Fibre Channel volumes can only be mounted as read/write once. Fibre Channel volumes support ownership management and SELinux relabeling.
Filesystem type to mount. Must be a filesystem type supported by the host operating system. Ex. "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" if unspecified.
-
-
-
lun
-
integer
-
Optional: FC target lun number
-
-
-
readOnly
-
boolean
-
Optional: Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts.
-
-
-
targetWWNs
-
Array< string >
-
Optional: FC target worldwide names (WWNs)
-
-
-
wwids
-
Array< string >
-
Optional: FC volume world wide identifiers (wwids) Either wwids or combination of targetWWNs and lun must be set, but not both simultaneously.
Driver is the name of the driver to use for this volume.
-
-
-
fsType
-
string
-
Filesystem type to mount. Must be a filesystem type supported by the host operating system. Ex. "ext4", "xfs", "ntfs". The default filesystem depends on FlexVolume script.
-
-
-
options
-
Map< string , string >
-
Optional: Extra command options if any.
-
-
-
readOnly
-
boolean
-
Optional: Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts.
Optional: SecretRef is reference to the secret object containing sensitive information to pass to the plugin scripts. This may be empty if no secret object is specified. If the secret object contains more than one secret, all secrets are passed to the plugin scripts.
Represents a Flocker volume mounted by the Flocker agent. One and only one of datasetName and datasetUUID should be set. Flocker volumes do not support ownership management or SELinux relabeling.
Represents a Persistent Disk resource in Google Compute Engine. A GCE PD must exist before mounting to a container. The disk must also be in the same GCE project and zone as the kubelet. A GCE PD can only be mounted as read/write once or read-only many times. GCE PDs support ownership management and SELinux relabeling.
Filesystem type of the volume that you want to mount. Tip: Ensure that the filesystem type is supported by the host operating system. Examples: "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" if unspecified. More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk
-
-
-
partition
-
integer
-
The partition in the volume that you want to mount. If omitted, the default is to mount by volume name. Examples: For volume /dev/sda1, you specify the partition as "1". Similarly, the volume partition for /dev/sda is "0" (or you can leave the property empty). More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk
-
-
-
pdName
-
string
-
Unique name of the PD resource in GCE. Used to identify the disk in GCE. More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk
-
-
-
readOnly
-
boolean
-
ReadOnly here will force the ReadOnly setting in VolumeMounts. Defaults to false. More info: https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk
Represents a volume that is populated with the contents of a git repository. Git repo volumes do not support ownership management. Git repo volumes support SELinux relabeling. DEPRECATED: GitRepo is deprecated. To provision a container with a git repo, mount an EmptyDir into an InitContainer that clones the repo using git, then mount the EmptyDir into the Pod's container.
Target directory name. Must not contain or start with '..'. If '.' is supplied, the volume directory will be the git repository. Otherwise, if specified, the volume will contain the git repository in the subdirectory with the given name.
EndpointsName is the endpoint name that details Glusterfs topology. More info: https://examples.k8s.io/volumes/glusterfs/README.md#create-a-pod
-
-
-
path
-
string
-
Path is the Glusterfs volume path. More info: https://examples.k8s.io/volumes/glusterfs/README.md#create-a-pod
-
-
-
readOnly
-
boolean
-
ReadOnly here will force the Glusterfs volume to be mounted with read-only permissions. Defaults to false. More info: https://examples.k8s.io/volumes/glusterfs/README.md#create-a-pod
Path of the directory on the host. If the path is a symlink, it will follow the link to the real path. More info: https://kubernetes.io/docs/concepts/storage/volumes#hostpath
-
-
-
type
-
string
-
Type for HostPath Volume Defaults to "" More info: https://kubernetes.io/docs/concepts/storage/volumes#hostpath
whether support iSCSI Discovery CHAP authentication
-
-
-
chapAuthSession
-
boolean
-
whether support iSCSI Session CHAP authentication
-
-
-
fsType
-
string
-
Filesystem type of the volume that you want to mount. Tip: Ensure that the filesystem type is supported by the host operating system. Examples: "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" if unspecified. More info: https://kubernetes.io/docs/concepts/storage/volumes#iscsi
-
-
-
initiatorName
-
string
-
Custom iSCSI Initiator Name. If initiatorName is specified with iscsiInterface simultaneously, new iSCSI interface : will be created for the connection.
-
-
-
iqn
-
string
-
Target iSCSI Qualified Name.
-
-
-
iscsiInterface
-
string
-
iSCSI Interface Name that uses an iSCSI transport. Defaults to 'default' (tcp).
-
-
-
lun
-
integer
-
iSCSI Target Lun number.
-
-
-
portals
-
Array< string >
-
iSCSI Target Portal List. The portal is either an IP or ip_addr:port if the port is other than default (typically TCP ports 860 and 3260).
-
-
-
readOnly
-
boolean
-
ReadOnly here will force the ReadOnly setting in VolumeMounts. Defaults to false.
Path that is exported by the NFS server. More info: https://kubernetes.io/docs/concepts/storage/volumes#nfs
-
-
-
readOnly
-
boolean
-
ReadOnly here will force the NFS export to be mounted with read-only permissions. Defaults to false. More info: https://kubernetes.io/docs/concepts/storage/volumes#nfs
-
-
-
server
-
string
-
Server is the hostname or IP address of the NFS server. More info: https://kubernetes.io/docs/concepts/storage/volumes#nfs
PersistentVolumeClaimVolumeSource references the user's PVC in the same namespace. This volume finds the bound PV and mounts that volume for the pod. A PersistentVolumeClaimVolumeSource is, essentially, a wrapper around another type of volume that is owned by someone else (the system).
ClaimName is the name of a PersistentVolumeClaim in the same namespace as the pod using this volume. More info: https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims
-
-
-
readOnly
-
boolean
-
Will force the ReadOnly setting in VolumeMounts. Default false.
Filesystem type to mount. Must be a filesystem type supported by the host operating system. Ex. "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" if unspecified.
-
-
-
pdID
-
string
-
ID that identifies Photon Controller persistent disk
FSType represents the filesystem type to mount Must be a filesystem type supported by the host operating system. Ex. "ext4", "xfs". Implicitly inferred to be "ext4" if unspecified.
-
-
-
readOnly
-
boolean
-
Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts.
Mode bits used to set permissions on created files by default. Must be an octal value between 0000 and 0777 or a decimal value between 0 and 511. YAML accepts both octal and decimal values, JSON requires decimal values for mode bits. Directories within the path are not affected by this setting. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set.
ReadOnly here will force the Quobyte volume to be mounted with read-only permissions. Defaults to false.
-
-
-
registry
-
string
-
Registry represents a single or multiple Quobyte Registry services specified as a string as host:port pair (multiple entries are separated with commas) which acts as the central registry for volumes
-
-
-
tenant
-
string
-
Tenant owning the given Quobyte volume in the Backend Used with dynamically provisioned Quobyte volumes, value is set by the plugin
-
-
-
user
-
string
-
User to map volume access to Defaults to serivceaccount user
-
-
-
volume
-
string
-
Volume is a string that references an already created Quobyte volume by name.
Filesystem type of the volume that you want to mount. Tip: Ensure that the filesystem type is supported by the host operating system. Examples: "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" if unspecified. More info: https://kubernetes.io/docs/concepts/storage/volumes#rbd
-
-
-
image
-
string
-
The rados image name. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it
-
-
-
keyring
-
string
-
Keyring is the path to key ring for RBDUser. Default is /etc/ceph/keyring. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it
-
-
-
monitors
-
Array< string >
-
A collection of Ceph monitors. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it
-
-
-
pool
-
string
-
The rados pool name. Default is rbd. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it
-
-
-
readOnly
-
boolean
-
ReadOnly here will force the ReadOnly setting in VolumeMounts. Defaults to false. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it
SecretRef is name of the authentication secret for RBDUser. If provided overrides keyring. Default is nil. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it
-
-
-
user
-
string
-
The rados user name. Default is admin. More info: https://examples.k8s.io/volumes/rbd/README.md#how-to-use-it
Adapts a Secret into a volume. The contents of the target Secret's Data field will be presented in a volume as files using the keys in the Data field as the file names. Secret volumes support ownership management and SELinux relabeling.
Optional: mode bits used to set permissions on created files by default. Must be an octal value between 0000 and 0777 or a decimal value between 0 and 511. YAML accepts both octal and decimal values, JSON requires decimal values for mode bits. Defaults to 0644. Directories within the path are not affected by this setting. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set.
If unspecified, each key-value pair in the Data field of the referenced Secret will be projected into the volume as a file whose name is the key and content is the value. If specified, the listed keys will be projected into the specified paths, and unlisted keys will not be present. If a key is specified which is not present in the Secret, the volume setup will error unless it is marked optional. Paths must be relative and may not contain the '..' path or start with '..'.
-
-
-
optional
-
boolean
-
Specify whether the Secret or its keys must be defined
-
-
-
secretName
-
string
-
Name of the secret in the pod's namespace to use. More info: https://kubernetes.io/docs/concepts/storage/volumes#secret
Filesystem type to mount. Must be a filesystem type supported by the host operating system. Ex. "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" if unspecified.
-
-
-
readOnly
-
boolean
-
Defaults to false (read/write). ReadOnly here will force the ReadOnly setting in VolumeMounts.
SecretRef specifies the secret to use for obtaining the StorageOS API credentials. If not specified, default values will be attempted.
-
-
-
volumeName
-
string
-
VolumeName is the human-readable name of the StorageOS volume. Volume names are only unique within a namespace.
-
-
-
volumeNamespace
-
string
-
VolumeNamespace specifies the scope of the volume within StorageOS. If no namespace is specified then the Pod's namespace will be used. This allows the Kubernetes name scoping to be mirrored within StorageOS for tighter integration. Set VolumeName to any name to override the default behaviour. Set to "default" if you are not using namespaces within StorageOS. Namespaces that do not pre-exist within StorageOS will be created.
Filesystem type to mount. Must be a filesystem type supported by the host operating system. Ex. "ext4", "xfs", "ntfs". Implicitly inferred to be "ext4" if unspecified.
-
-
-
storagePolicyID
-
string
-
Storage Policy Based Management (SPBM) profile ID associated with the StoragePolicyName.
-
-
-
storagePolicyName
-
string
-
Storage Policy Based Management (SPBM) profile name.
key is the label key that the selector applies to.
-
-
-
operator
-
string
-
operator represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists and DoesNotExist.
-
-
-
values
-
Array< string >
-
values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. This array is replaced during a strategic merge patch.
Selects a field of the pod: supports metadata.name, metadata.namespace, metadata.labels['<KEY>'], metadata.annotations['<KEY>'], spec.nodeName, spec.serviceAccountName, status.hostIP, status.podIP, status.podIPs.
Selects a resource of the container: only resources limits and requests (limits.cpu, limits.memory, limits.ephemeral-storage, requests.cpu, requests.memory and requests.ephemeral-storage) are currently supported.
ConfigMapEnvSource selects a ConfigMap to populate the environment variables with. The contents of the target ConfigMap's Data field will represent the key-value pairs as environment variables.
SecretEnvSource selects a Secret to populate the environment variables with. The contents of the target Secret's Data field will represent the key-value pairs as environment variables.
LifecycleHandler defines a specific action that should be taken in a lifecycle hook. One and only one of the fields, except TCPSocket must be specified.
Deprecated. TCPSocket is NOT supported as a LifecycleHandler and kept for the backward compatibility. There are no validation of this field and lifecycle hooks will fail in runtime when tcp handler is specified.
Command is the command line to execute inside the container, the working directory for the command is root ('/') in the container's filesystem. The command is simply exec'd, it is not run inside a shell, so traditional shell instructions ('
Port number of the gRPC service. Number must be in the range 1 to 65535.
-
-
-
service
-
string
-
Service is the name of the service to place in the gRPC HealthCheckRequest (see https://github.com/grpc/grpc/blob/master/doc/health-checking.md). If this is not specified, the default behavior is defined by gRPC.
Name or number of the port to access on the container. Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.
-
-
-
scheme
-
string
-
Scheme to use for connecting to the host. Defaults to HTTP. Possible enum values: - "HTTP" means that the scheme used will be http:// - "HTTPS" means that the scheme used will be https://
Quantity is a fixed-point representation of a number. It provides convenient marshaling/unmarshaling in JSON and YAML, in addition to String() and AsInt64() accessors. The serialization format is: ::= (Note that may be empty, from the "" case in .) ::= 0 | 1 | ... | 9 ::= | ::= | . | . | . ::= "+" | "-" ::= | ::= | | ::= Ki | Mi | Gi | Ti | Pi | Ei (International System of units; See: http://physics.nist.gov/cuu/Units/binary.html) ::= m | "" | k | M | G | T | P | E (Note that 1024 = 1Ki but 1000 = 1k; I didn't choose the capitalization.) ::= "e" | "E" No matter which of the three exponent forms is used, no quantity may represent a number greater than 2^63-1 in magnitude, nor may it have more than 3 decimal places. Numbers larger or more precise will be capped or rounded up. (E.g.: 0.1m will rounded up to 1m.) This may be extended in the future if we require larger or smaller quantities. When a Quantity is parsed from a string, it will remember the type of suffix it had, and will use the same type again when it is serialized. Before serializing, Quantity will be put in "canonical form". This means that Exponent/suffix will be adjusted up or down (with a corresponding increase or decrease in Mantissa) such that: a. No precision is lost b. No fractional digits will be emitted c. The exponent (or suffix) is as large as possible. The sign will be omitted unless the number is negative. Examples: 1.5 will be serialized as "1500m" 1.5Gi will be serialized as "1536Mi" Note that the quantity will NEVER be internally represented by a floating point number. That is the whole point of this exercise. Non-canonical values will still parse as long as they are well formed, but will be re-emitted in their canonical form. (So always use canonical form, or don't diff.) This format is intended to make it difficult to use these numbers without writing some sort of special handling code in the hopes that that will cause implementors to also use a fixed point implementation.
FieldsV1 stores a set of fields in a data structure like a Trie, in JSON format. Each key is either a '.' representing the field itself, and will always map to an empty set, or a string representing a sub-field or item. The string will follow one of these four formats: 'f:', where is the name of a field in a struct, or key in a map 'v:', where is the exact json formatted value of a list item 'i:', where is position of a item in a list 'k:', where is a map of a list item's key fields to their unique values If a key maps to an empty Fields value, the field that key represents is part of the set. The exact format is defined in sigs.k8s.io/structured-merge-diff
An empty preferred scheduling term matches all objects with implicit weight 0 (i.e. it's a no-op). A null preferred scheduling term matches no objects (i.e. is also a no-op).
A node selector represents the union of the results of one or more label queries over a set of nodes; that is, it represents the OR of the selectors represented by the node selector terms.
Defines a set of pods (namely those matching the labelSelector relative to the given namespace(s)) that this pod should be co-located (affinity) or not co-located (anti-affinity) with, where co-located is defined as running on a node whose value of the label with key matches that of any node on which a pod of the set of pods is running
A label query over the set of namespaces that the term applies to. The term is applied to the union of the namespaces selected by this field and the ones listed in the namespaces field. null selector and null or empty namespaces list means "this pod's namespace". An empty selector ({}) matches all namespaces. This field is beta-level and is only honored when PodAffinityNamespaceSelector feature is enabled.
-
-
-
namespaces
-
Array< string >
-
namespaces specifies a static list of namespace names that the term applies to. The term is applied to the union of the namespaces listed in this field and the ones selected by namespaceSelector. null or empty namespaces list and null namespaceSelector means "this pod's namespace"
-
-
-
topologyKey
-
string
-
This pod should be co-located (affinity) or not co-located (anti-affinity) with the pods matching the labelSelector in the specified namespaces, where co-located is defined as running on a node whose value of the label with key topologyKey matches that of any node on which any of the selected pods is running. Empty topologyKey is not allowed.
APIGroup is the group for the resource being referenced. If APIGroup is not specified, the specified Kind must be in the core API group. For any other third-party types, APIGroup is required.
Last time the condition transitioned from one status to another.
-
-
-
message
-
string
-
Human-readable message indicating details about last transition.
-
-
-
reason
-
string
-
Unique, this should be a short, machine understandable string that gives the reason for condition's last transition. If it reports "ResizeStarted" that means the underlying persistent volume is being resized.
-
-
-
status
-
string
-
No description available
-
-
-
type
-
string
-
Possible enum values: - "FileSystemResizePending" - controller resize is finished and a file system resize is pending on node - "Resizing" - a user trigger resize of pvc has been started
Optional: mode bits used to set permissions on this file. Must be an octal value between 0000 and 0777 or a decimal value between 0 and 511. YAML accepts both octal and decimal values, JSON requires decimal values for mode bits. If not specified, the volume defaultMode will be used. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set.
-
-
-
path
-
string
-
The relative path of the file to map the key to. May not be an absolute path. May not contain the path element '..'. May not start with the string '..'.
Required: Selects a field of the pod: only annotations, labels, name and namespace are supported.
-
-
-
mode
-
integer
-
Optional: mode bits used to set permissions on this file, must be an octal value between 0000 and 0777 or a decimal value between 0 and 511. YAML accepts both octal and decimal values, JSON requires decimal values for mode bits. If not specified, the volume defaultMode will be used. This might be in conflict with other options that affect the file mode, like fsGroup, and the result can be other mode bits set.
-
-
-
path
-
string
-
Required: Path is the relative path name of the file to be created. Must not be absolute or contain the '..' path. Must be utf-8 encoded. The first item of the relative path must not start with '..'
Selects a resource of the container: only resources limits and requests (limits.cpu, limits.memory, requests.cpu and requests.memory) are currently supported.
May contain labels and annotations that will be copied into the PVC when creating it. No other fields are allowed and will be rejected during validation.
The specification for the PersistentVolumeClaim. The entire content is copied unchanged into the PVC that gets created from this template. The same fields as in a PersistentVolumeClaim are also valid here.
A null or empty node selector term matches no objects. The requirements of them are ANDed. The TopologySelectorTerm type implements a subset of the NodeSelectorTerm.
Adapts a ConfigMap into a projected volume. The contents of the target ConfigMap's Data field will be presented in a projected volume as files using the keys in the Data field as the file names, unless the items element is populated with specific mappings of keys to paths. Note that this is identical to a configmap volume source without the default mode.
If unspecified, each key-value pair in the Data field of the referenced ConfigMap will be projected into the volume as a file whose name is the key and content is the value. If specified, the listed keys will be projected into the specified paths, and unlisted keys will not be present. If a key is specified which is not present in the ConfigMap, the volume setup will error unless it is marked optional. Paths must be relative and may not contain the '..' path or start with '..'.
-
-
-
name
-
string
-
Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
-
-
-
optional
-
boolean
-
Specify whether the ConfigMap or its keys must be defined
Represents downward API info for projecting into a projected volume. Note that this is identical to a downwardAPI volume source without the default mode.
Adapts a secret into a projected volume. The contents of the target Secret's Data field will be presented in a projected volume as files using the keys in the Data field as the file names. Note that this is identical to a secret volume source without the default mode.
If unspecified, each key-value pair in the Data field of the referenced Secret will be projected into the volume as a file whose name is the key and content is the value. If specified, the listed keys will be projected into the specified paths, and unlisted keys will not be present. If a key is specified which is not present in the Secret, the volume setup will error unless it is marked optional. Paths must be relative and may not contain the '..' path or start with '..'.
-
-
-
name
-
string
-
Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
-
-
-
optional
-
boolean
-
Specify whether the Secret or its key must be defined
ServiceAccountTokenProjection represents a projected service account token volume. This projection can be used to insert a service account token into the pods runtime filesystem for use against APIs (Kubernetes API Server or otherwise).
Audience is the intended audience of the token. A recipient of a token must identify itself with an identifier specified in the audience of the token, and otherwise should reject the token. The audience defaults to the identifier of the apiserver.
-
-
-
expirationSeconds
-
integer
-
ExpirationSeconds is the requested duration of validity of the service account token. As the token approaches expiration, the kubelet volume plugin will proactively rotate the service account token. The kubelet will start trying to rotate the token if the token is older than 80 percent of its time to live or if the token is older than 24 hours.Defaults to 1 hour and must be at least 10 minutes.
-
-
-
path
-
string
-
Path is the path relative to the mount point of the file to project the token into.
Represents a key's relationship to a set of values. Valid operators are In, NotIn, Exists, DoesNotExist. Gt, and Lt. Possible enum values: - "DoesNotExist" - "Exists" - "Gt" - "In" - "Lt" - "NotIn"
-
-
-
values
-
Array< string >
-
An array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty. If the operator is Gt or Lt, the values array must have a single element, which will be interpreted as an integer. This array is replaced during a strategic merge patch.
Before v3.0, only one controller could run at once. (If it crashed, Kubernetes would start another pod.)
-
-
v3.0
-
-
For many users, a short loss of workflow service may be acceptable - the new controller will just continue running
-workflows if it restarts. However, with high service guarantees, new pods may take too long to start running workflows.
-You should run two replicas, and one of which will be kept on hot-standby.
-
A voluntary pod disruption can cause both replicas to be replaced at the same time. You should use a Pod Disruption
-Budget to prevent this and Pod Priority to recover faster from an involuntary pod disruption:
HTTP Template is a type of template which can execute HTTP Requests.
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:http-template-
-spec:
-entrypoint:main
-templates:
--name:main
-steps:
---name:get-google-homepage
-template:http
-arguments:
-parameters:[{name:url, value:"https://www.google.com"}]
--name:http
-inputs:
-parameters:
--name:url
-http:
-timeoutSeconds:20# Default 30
-url:"{{inputs.parameters.url}}"
-method:"GET"# Default GET
-headers:
--name:"x-header-name"
-value:"test-value"
-# Template will succeed if evaluated to true, otherwise will fail
-# Available variables:
-# request.body: string, the request body
-# request.headers: map[string][]string, the request headers
-# response.url: string, the request url
-# response.method: string, the request method
-# response.statusCode: int, the response status code
-# response.body: string, the response body
-# response.headers: map[string][]string, the response headers
-successCondition:"response.bodycontains\"google\""# available since v3.3
-body:"testbody"# Change request body
-
HTTP Templates use the Argo Agent, which executes the requests independently of the controller. The Agent and the Workflow
-Controller communicate through the WorkflowTaskSet CRD, which is created for each running Workflow that requires the use
-of the Agent.
-
In order to use the Argo Agent, you will need to ensure that you have added the appropriate workflow RBAC to add an agent role with to Argo Workflows. An example agent role can be found in the quick-start manifests.
If you have the JetBrains Kubernetes Plugin
-installed in your IDE, the validation can be configured in the Kubernetes plugin settings
-instead of using the internal JSON schema file validator.
-
-
Unlike the previous JSON schema validation method, the plugin detects the necessary validation
-based on Kubernetes resource definition keys and does not require a file glob pattern.
-Like the previously described method:
Argo Workflows is an open source container-native workflow engine for orchestrating parallel jobs on Kubernetes. Argo
-Workflows is implemented as a Kubernetes CRD (Custom Resource Definition).
-
-
Define workflows where each step in the workflow is a container.
-
Model multi-step workflows as a sequence of tasks or capture the dependencies between tasks using a directed acyclic
- graph (DAG).
-
Easily run compute intensive jobs for machine learning or data processing in a fraction of the time using Argo
- Workflows on Kubernetes.
We host monthly community meetings where we and the community showcase demos and discuss the current and future state of
-the project. Feel free to join us! For Community Meeting information, minutes and recordings
-please see here.
-
Participation in the Argo Workflows project is governed by
-the CNCF Code of Conduct
If you just want to try out Argo Workflows in a non-production environment (including on desktop via minikube/kind/k3d etc) follow the quick-start guide.
To install Argo Workflows, navigate to the releases page and find the release you wish to use (the latest full release is preferred). Scroll down to the Controller and Server section and execute the kubectl commands.
-
You can use Kustomize to patch your preferred configurations on top of the base manifest.
-
⚠️ If you are using GitOps, never use Kustomize remote base: this is dangerous. Instead, copy the manifests into your Git repo.
-
⚠️ latest is tip, not stable. Never run it in production.
A cluster install will watch and execute workflows in all namespaces. This is the default installation option when installing using the official release manifests.
-
A namespace install only executes workflows in the namespace it is installed in (typically argo). Look for namespace-install.yaml in the release assets.
-
A managed namespace install: only executes workflows in a separate namespace from the one it is installed in. See Managed Namespace for more details.
Traditionally, Argo workflows has supported input parameters from UI only when the workflow starts,
-and after that, it's pretty much on autopilot. But, there are a lot of use cases where human interaction is required.
-
This interaction is in the form of providing input text in the middle of the workflow, choosing from a dropdown of the options which a workflow step itself is intelligently generating.
A human approval before doing something in production environment.
-
Programmatic generation of a list of inputs from which the user chooses.
-Choosing from a list of available databases which the workflow itself is generating.
-
-
This feature is achieved via suspend template.
-
The workflow will pause at a Suspend node, and user will be able to update parameters using fields type text or dropdown.
A key-only artifact is an input or output artifact where you only specify the key, omitting the bucket, secrets etc. When these are omitted, the bucket/secrets from the configured artifact repository is used.
-
This allows you to move the configuration of the artifact repository out of the workflow specification.
-
This is closely related to artifact repository ref. You'll want to use them together for maximum benefit.
-
This should probably be your default if you're using v3.0:
-
-
Reduces the size of workflows (improved performance).
-
User owned artifact repository set-up configuration (simplified management).
-
Decouples the artifact location configuration from the workflow. Allowing you to re-configure the artifact repository without changing your workflows or templates.
You can also create Workflows directly with kubectl.
-However, the Argo CLI offers extra features that kubectl does not, such as YAML validation, workflow visualization, parameter passing, retries and resubmits, suspend and resume, and more.
A LifecycleHook triggers an action based on a conditional expression or on completion of a step or template. It is configured either at the workflow-level or template-level, for instance as a function of the workflow.status or steps.status, respectively. A LifecycleHook executes during execution time and executes once. It will execute in parallel to its step or template once the expression is satisfied.
-
In other words, a LifecycleHook functions like an exit handler with a conditional expression. You must not name a LifecycleHookexit or it becomes an exit handler; otherwise the hook name has no relevance.
-
Workflow-level LifecycleHook: Executes the template when a configured expression is met during the workflow.
outputs are not usable since LifecycleHook executes during execution time and outputs are not produced until the step is completed. You can use outputs from previous steps, just not the one you're hooking into. If you'd like to use outputs create an exit handler instead - all the status variable are available there so you can still conditionally decide what to do.
Put differently, an exit handler is like a workflow-level LifecycleHook with an expression of workflow.status == "Succeeded" or workflow.status == "Failed" or workflow.status == "Error".
You can configure Argo Server to show custom links:
-
-
A "Get Help" button in the bottom right of the window linking to you organization help pages or chat room.
-
Deep-links to your facilities (e.g. logging facility) in the UI for both the workflow and each workflow pod.
-
Adds a button to the top of workflow view to navigate to customized views.
-
-
Links can contain placeholder variables. Placeholder variables are indicated by the dollar sign and curly braces: ${variable}.
-
These are the commonly used variables:
-
-
${metadata.namespace}: Kubernetes namespace of the current workflow / pod / event source / sensor
-
${metadata.name}: Name of the current workflow / pod / event source / sensor
-
${status.startedAt}: Start time-stamp of the workflow / pod, in the format of 2021-01-01T10:35:56Z
-
${status.finishedAt}: End time-stamp of the workflow / pod, in the format of 2021-01-01T10:35:56Z. If the workflow/pod is still running, this variable will be null
Epoch time-stamps are available now. These are useful if we want to add links to logging facilities like Grafana
-or DataDog, as they support Unix epoch time-stamp formats as URL
-parameters:
-
-
${status.startedAtEpoch}: Start time-stamp of the workflow/pod, in the Unix epoch time format in milliseconds, e.g. 1609497000000.
-
${status.finishedAtEpoch}: End time-stamp of the workflow/pod, in the Unix epoch time format in milliseconds, e.g. 1609497000000. If the workflow/pod is still running, this variable will represent the current time.
-
-
-
v3.1 and after
-
-
In addition to the above variables, we can now access all workflow fields under ${workflow}.
-
For example, one may find it useful to define a custom label in the workflow and access it by ${workflow.metadata.labels.custom_label_name}
-
We can also access workflow fields in a pod link. For example, ${workflow.metadata.name} returns
-the name of the workflow instead of the name of the pod. If the field doesn't exist on the workflow then the value will be an empty string.
You can install Argo in either namespace scoped or cluster scoped configurations.
-The main difference is whether you install Roles or ClusterRoles, respectively.
-
In namespace scoped configuration, you must run both the Workflow Controller and Argo Server using --namespaced.
-If you want to run workflows in a separate namespace, add --managed-namespace as well.
-(In cluster scoped configuration, don't include --namespaced or --managed-namespace.)
Please note that both cluster scoped and namespace scoped configurations require "admin" roles to install because Argo's Custom Resource Definitions (CRDs) must be created (CRDs are cluster scoped objects).
-
-
Example Use Case
-
You can use a managed namespace install if you want some users or services to run Workflows without granting them privileges in the namespace where Argo Workflows is installed.
-For example, if you only run CI/CD Workflows that are maintained by the same team that manages the Argo Workflows installation, you may want a namespace install.
-But if all the Workflows are run by a separate data science team, you may want to give them a "data-science-workflows" namespace and use a managed namespace install of Argo Workflows in another namespace.
Workflows often have outputs that are expensive to compute.
-Memoization reduces cost and workflow execution time by recording the result of previously run steps:
-it stores the outputs of a template into a specified cache with a variable key.
-
Prior to version 3.5 memoization only works for steps which have outputs, if you attempt to use it on steps which do not it should not work (there are some cases where it does, but they shouldn't). It was designed for 'pure' steps, where the purpose of running the step is to calculate some outputs based upon the step's inputs, and only the inputs. Pure steps should not interact with the outside world, but workflows won't enforce this on you.
-
If you are using workflows prior to version 3.5 you should look at the work avoidance technique instead of memoization if your steps don't have outputs.
-
In version 3.5 or later all steps can be memoized, whether or not they have outputs.
Currently, the cached data is stored in config-maps.
-This allows you to easily manipulate cache entries manually through kubectl and the Kubernetes API without having to go through Argo.
-All cache config-maps must have the label workflows.argoproj.io/configmap-type: Cache to be used as a cache. This prevents accidental access to other important config-maps in the system
Memoization is set at the template level. You must specify a key, which can be static strings but more often depend on inputs.
-You must also specify a name for the config-map cache.
-Optionally you can set a maxAge in seconds or hours (e.g. 180s, 24h) to define how long should it be considered valid. If an entry is older than the maxAge, it will be ignored.
In order to use memoization it is necessary to add the verbs create and update to the configmaps resource for the appropriate (cluster) roles. In the case of a cluster install the argo-cluster-role cluster role should be updated, whilst for a namespace install the argo-role role should be updated.
If you see errors like error creating cache entry: ConfigMap \"reuse-task\" is invalid: []: Too long: must have at most 1048576 characters,
- this is due to the 1MB limit placed on the size of ConfigMap.
- Here are a couple of ways that might help resolve this:
-
Delete the existing ConfigMap cache or switch to use a different cache.
-
Reduce the size of the output parameters for the nodes that are being memoized.
-
Split your cache into different memoization keys and cache names so that each cache entry is small.
-
-
-
My step isn't getting memoized, why not?
- If you are running workflows <3.5 ensure that you have specified at least one output on the step.
Argo emits a certain number of controller metrics that inform on the state of the controller at any given time. Furthermore,
-users can also define their own custom metrics to inform on the state of their Workflows.
-
Custom Prometheus metrics can be defined to be emitted on a Workflow- and Template-level basis. These can be useful
-for many cases; some examples:
-
-
Keeping track of the duration of a Workflow or Template over time, and setting an alert if it goes beyond a threshold
-
Keeping track of the number of times a Workflow or Template fails over time
-
Reporting an important internal metric, such as a model training score or an internal error rate
-
-
Emitting custom metrics with Argo is easy, but it's important to understand what makes a good Prometheus metric and the
-best way to define metrics in Argo to avoid problems such as cardinality explosion.
Metrics that inform on the state of the controller; i.e., they answer the question "What is the state of the controller right now?"
-Default controller metrics can be scraped from service workflow-controller-metrics at the endpoint <host>:9090/metrics
Metrics that inform on the state of a Workflow, or a series of Workflows. These custom metrics are defined by the user in the Workflow spec.
-
Emitting custom metrics is the responsibility of the emitter owner. Since the user defines Workflows in Argo, the user is responsible
-for emitting metrics correctly.
Prometheus metrics should be thought of as ephemeral data points of running processes; i.e., they are the answer to
-the question "What is the state of my system right now?". Metrics should report things such as:
-
-
a counter of the number of times a workflow or steps has failed, or
-
a gauge of workflow duration, or
-
an average of an internal metric such as a model training score or error rate.
-
-
Metrics are then routinely scraped and stored and -- when they are correctly designed -- they can represent time series.
-Aggregating the examples above over time could answer useful questions such as:
-
-
How has the error rate of this workflow or step changed over time?
-
How has the duration of this workflow changed over time? Is the current workflow running for too long?
-
Is our model improving over time?
-
-
Prometheus metrics should not be thought of as a store of data. Since metrics should only report the state of the system
-at the current time, they should not be used to report historical data such as:
-
-
the status of an individual instance of a workflow, or
-
how long a particular instance of a step took to run.
-
-
Metrics are also ephemeral, meaning there is no guarantee that they will be persisted for any amount of time. If you need
-a way to view and analyze historical data, consider the workflow archive or reporting to logs.
Number of workflow in each phase. The Running count does not mean that a workflows pods are running, just that the controller has scheduled them. A workflow can be stuck in Running with pending pods for a long time.
A histogram of durations of operations. An operation is a single workflow reconciliation loop within the workflow-controller. It's the time for the controller to process a single workflow after it has been read from the cluster and is a measure of the performance of the controller affected by the complexity of the workflow.
It is possible for a workflow to start, but no pods be running (e.g. cluster is too busy to run them). This metric sheds light on actual work being done.
In order to analyze the behavior of a workflow over time, we need to be able to link different instances
-(i.e. individual executions) of a workflow together into a "series" for the purposes of emitting metrics. We do so by linking them together
-with the same metric descriptor.
-
In Prometheus, a metric descriptor is defined as a metric's name and its key-value labels. For example, for a metric
-tracking the duration of model execution over time, a metric descriptor could be:
This metric then represents the amount of time that "Model A" took to train in the phase "Validation". It is important
-to understand that the metric name and its labels form the descriptor: argo_workflows_model_exec_time{model_name="model_b",phase="validation"}
-is a different metric (and will track a different "series" altogether).
-
Now, whenever we run our first workflow that validates "Model A" a metric with the amount of time it took it to do so will
-be created and emitted. For each subsequent time that this happens, no new metrics will be emitted and the same metric
-will be updated with the new value. Since, in effect, we are interested on the execution time of "validation" of "Model A"
-over time, we are no longer interested in the previous metric and can assume it has already been scraped.
-
In summary, whenever you want to track a particular metric over time, you should use the same metric name and metric
-labels wherever it is emitted. This is how these metrics are "linked" as belonging to the same series.
Metrics are defined in-place on the Workflow/Step/Task where they are emitted from. Metrics are always processed after
-the Workflow/Step/Task completes, with the exception of real-time metrics.
-
Metric definitions must include a name and a help doc string. They can also include any number of labels (when
-defining labels avoid cardinality explosion). Metrics with the same namemust always use the same exact help string,
-having different metrics with the same name, but with a different help string will cause an error (this is a Prometheus requirement).
-
All metrics can also be conditionally emitted by defining a when clause. This when clause works the same as elsewhere
-in a workflow.
-
A metric must also have a type, it can be one of gauge, histogram, and counter (see below). Within
-the metric type a value must be specified. This value can be either a literal value of be an Argo variable.
-
When defining a histogram, buckets must also be provided (see below).
-
Argo variables can be included anywhere in the metric spec, such as in labels, name, help, when, etc.
-
Metric names can only contain alphanumeric characters, _, and :.
In Argo you can define a metric on the Workflow level or on the Template level. Here is an example of a Workflow
-level Gauge metric that will report the Workflow duration time:
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:model-training-
-spec:
-entrypoint:steps
-metrics:
-prometheus:
--name:exec_duration_gauge# Metric name (will be prepended with "argo_workflows_")
-labels:# Labels are optional. Avoid cardinality explosion.
--key:name
-value:model_a
-help:"Durationgaugebyname"# A help doc describing your metric. This is required.
-gauge:# The metric type. Available are "gauge", "histogram", and "counter".
-value:"{{workflow.duration}}"# The value of your metric. It could be an Argo variable (see variables doc) or a literal value
-
-...
-
-
An example of a Template-level Counter metric that will increase a counter every time the step fails:
-
...
-templates:
--name:flakey
-metrics:
-prometheus:
--name:result_counter
-help:"Countofstepexecutionbyresultstatus"
-labels:
--key:name
-value:flakey
-when:"{{status}}==Failed"# Emit the metric conditionally. Works the same as normal "when"
-counter:
-value:"1"# This increments the counter by 1
-container:
-image:python:alpine3.6
-command:["python",-c]
-# fail with a 66% probability
-args:["importrandom;importsys;exit_code=random.choice([0,1,1]);sys.exit(exit_code)"]
-...
-
-
A similar example of such a Counter metric that will increase for every step status
-
...
-templates:
--name:flakey
-metrics:
-prometheus:
--name:result_counter
-help:"Countofstepexecutionbyresultstatus"
-labels:
--key:name
-value:flakey
--key:status
-value:"{{status}}"# Argo variable in `labels`
-counter:
-value:"1"
-container:
-image:python:alpine3.6
-command:["python",-c]
-# fail with a 66% probability
-args:["importrandom;importsys;exit_code=random.choice([0,1,1]);sys.exit(exit_code)"]
-...
-
-
Finally, an example of a Template-level Histogram metric that tracks an internal value:
-
...
-templates:
--name:random-int
-metrics:
-prometheus:
--name:random_int_step_histogram
-help:"Valueoftheintemittedbyrandom-intatsteplevel"
-when:"{{status}}==Succeeded"# Only emit metric when step succeeds
-histogram:
-buckets:# Bins must be defined for histogram metrics
--2.01# and are part of the metric descriptor.
--4.01# All metrics in this series MUST have the
--6.01# same buckets.
--8.01
--10.01
-value:"{{outputs.parameters.rand-int-value}}"# References itself for its output (see variables doc)
-outputs:
-parameters:
--name:rand-int-value
-globalName:rand-int-value
-valueFrom:
-path:/tmp/rand_int.txt
-container:
-image:alpine:latest
-command:[sh,-c]
-args:["RAND_INT=$((1+RANDOM%10));echo$RAND_INT;echo$RAND_INT>/tmp/rand_int.txt"]
-...
-
Argo supports a limited number of real-time metrics. These metrics are emitted in real-time, beginning when the step execution starts
-and ending when it completes. Real-time metrics are only available on Gauge type metrics and with a limited number of variables.
-
To define a real-time metric simply add realtime: true to a gauge metric with a valid real-time variable. For example:
By default, metrics are emitted by the workflow-controller on port 9090 on the /metrics path. By port-forwarding to the pod you can view the metrics in your browser at http://localhost:9090/metrics:
If you have more than one controller pod, using one as a hot-standby, you should use a headless service to ensure that each pod is being scraped so that no metrics are missed.
metricsConfig:|
-# Enabled controls metric emission. Default is true, set "enabled: false" to turn off
-enabled: true
-
-# Path is the path where metrics are emitted. Must start with a "/". Default is "/metrics"
-path: /metrics
-
-# Port is the port where metrics are emitted. Default is "9090"
-port: 8080
-
-# MetricsTTL sets how often custom metrics are cleared from memory. Default is "0", metrics are never cleared
-metricsTTL: "10m"
-
-# IgnoreErrors is a flag that instructs prometheus to ignore metric emission errors. Default is "false"
-ignoreErrors: false
-
-# Use a self-signed cert for TLS, default false
-secure: false
-
The resume, stop and retry Argo CLI and API commands support a --node-field-selector parameter to allow the user to select a subset of nodes for the command to apply to.
-
In the case of the resume and stop commands these are the nodes that should be resumed or stopped.
-
In the case of the retry command it allows specifying nodes that should be restarted even if they were previously successful (and must be used in combination with --restart-successful)
Display name of the node. This is the name of the node as it is displayed on the CLI or UI, without considering its ancestors (see example below). This is a useful shortcut if there is only one node with the same displayName
-
-
-
name
-
Full name of the node. This is the full name of the node, including its ancestors (see example below). Using name is necessary when two or more nodes share the same displayName and disambiguation is required.
-
-
-
templateName
-
Template name of the node
-
-
-
phase
-
Phase status of the node - e.g. Running
-
-
-
templateRef.name
-
The name of the workflow template the node is referring to
-
-
-
templateRef.template
-
The template within the workflow template the node is referring to
-
-
-
inputs.parameters.<NAME>.value
-
The value of input parameter NAME
-
-
-
-
The operator can be '=' or '!='. Multiple selectors can be combined with a comma, in which case they are anded together.
Here we have two steps with the same displayName: wait-approval. To select one to suspend, we need to use their
-name, either appr-promotion-ffsv4.app1.wait-approval or appr-promotion-ffsv4.app3.wait-approval. If it is not clear
-what the full name of a node is, it can be found using kubectl:
-
$kubectlgetwfappr-promotion-ffsv4-oyaml
-
-...
-appr-promotion-ffsv4-3235686597:
-boundaryID:appr-promotion-ffsv4-3079407832
-displayName:wait-approval# <- Display Name
-finishedAt:null
-id:appr-promotion-ffsv4-3235686597
-name:appr-promotion-ffsv4.app1.wait-approval# <- Full Name
-phase:Running
-startedAt:"2021-01-20T17:00:25Z"
-templateRef:
-name:sample-template
-template:waiting-for-approval
-templateScope:namespaced/wftempl1
-type:Suspend
-...
-
Argo stores workflows as Kubernetes resources (i.e. within EtcD). This creates a limit to their size as resources must be under 1MB. Each resource includes the status of each node, which is stored in the /status/nodes field for the resource. This can be over 1MB. If this happens, we try and compress the node status and store it in /status/compressedNodes. If the status is still too large, we then try and store it in an SQL database.
-
To enable this feature, configure a Postgres or MySQL database under persistence in your configuration and set nodeStatusOffLoad: true.
Even after compressing node statuses, the workflow exceeded the EtcD
-size limit. To resolve, either enable node status offload as described
-above or look for ways to reduce the size of your workflow manifest:
-
-
Use withItems or withParams to consolidate similar templates into a single parametrized template
-
Use template defaults to factor shared template options to the workflow level
-
Use workflow templates to factor frequently-used templates into separate resources
-
Use workflows of workflows to factor a large workflow into a workflow of smaller workflows
Pods in a workflow can report their own progress during their runtime. This self reported progress overrides the
-auto-generated progress.
-
Reporting progress works as follows:
-
-
create and write the progress to a file indicated by the env variable ARGO_PROGRESS_FILE
-
format of the progress must be N/M
-
-
The executor will read this file every 3s and if there was an update,
-patch the pod annotations with workflows.argoproj.io/progress: N/M.
-The controller picks this up and writes the progress to the appropriate Status properties.
-
Initially the progress of a workflows' pod is always 0/1. If you want to influence this, make sure to set an initial
-progress annotation on the pod:
The motivation for this is to enable users to automatically have certain Artifacts specified to be automatically garbage collected.
-
Artifacts can be specified for Garbage Collection at different stages: OnWorkflowCompletion, OnWorkflowDeletion, OnWorkflowSuccess, OnWorkflowFailure, or Never
WorkflowSpec has an ArtifactGC structure, which consists of an ArtifactGCStrategy, as well as the optional designation of a ServiceAccount and Pod metadata (labels and annotations) to be used by the Pod doing the deletion. The ArtifactGCStrategy can be set to OnWorkflowCompletion, OnWorkflowDeletion, OnWorkflowSuccess, OnWorkflowFailure, or Never
-
Artifact has an ArtifactGC section which can be used to override the Workflow level.
For each ArtifactGCStrategy the Controller will execute one Pod that runs in the user's namespace and deletes all artifacts pertaining to that strategy.
-
-
Since OnWorkflowSuccess happens at the same time as OnWorkflowCompletion and OnWorkflowFailure also happens at the same time as OnWorkflowCompletion, we can consider consolidating these GC Strategies together.
-
We will have a new CRD type called ArtifactGCTask and use one or more of them to specify the Artifacts which the GC Pod will read and then write Status to (note individual artifacts have individual statuses). The Controller will read the Status and reflect that in the Workflow Status. The Controller will deem the ArtifactGCTasks ready to read once the Pod has completed (in success or failure).
-
Once the GC Pod has completed and the Workflow status has been persisted, assuming the Pod completed with Success, the Controller can delete the ArtifactGCTasks, which will cause the GC Pod to also get deleted as it will be "owned" by the ArtifactGCTasks.
-
The Workflow will have a Finalizer on it to prevent it from being deleted until Artifact GC has occurred. Once all deletions for all GC Strategies have occurred, the Controller will remove the Finalizer.
If a deletion fails, the Pod will retry a few times through exponential back off. Note: it will not be considered a failure if the key does not exist - the principal of idempotence will allow this (i.e. if a Pod were to get evicted and then re-run it should be okay if some artifacts were previously deleted).
-
Once it retries a few times, if it didn't succeed, it will end in a "Failed" state. The user will manually need to delete the ArtifactGCTasks (which will delete the GC Pod), and remove the Finalizer on the Workflow.
-
The Failure will be reflected in both the Workflow Conditions as well as as a Kubernetes Event (and the Artifacts that failed will have "Deleted"=false).
For reference, these slides were presented to the Argo Contributor meeting on 7/12/22 which go through some of the alternative options that were weighed. These alternatives are explained below:
The POC that was done, which uses just one Pod to delete each Artifact, was considered as an alternative for MVP (Option 1 from the slides).
-
This option has these benefits:
-
-
simpler in that the Pod doesn't require any additional Object to report status (e.g. ArtifactGCTask) because it simply succeeds or fails based on its exit code (whereas in Option 2 the Pod needs to report individual failure statuses for each artifact)
-
could have a very minimal Service Account which provides access to just that one artifact's location
-
-
and these drawbacks:
-
-
deletion is slower when performed by multiple Pods
-
a Workflow with thousands of artifacts causes thousands of Pods to get executed, which could overwhelm kube-scheduler and kube-apiserver.
-
if we delay the Artifact GC Pods by giving them a lower priority than the Workflow Pods, users will not get their artifacts deleted when they expect and may log bugs
-
-
Summarizing ADR statement:
-"In the context of Artifact Garbage Collection, facing whether to use a separate Pod for every artifact or not, we decided not to, to achieve faster garbage collection and reduced load on K8S, accepting that we will require a new CRD type."
We considered some alternatives for how to specify Service Account and/or Annotations, which are applied to give the GC Pod access (slide 12). We will have them specify this information in a new ArtifactGC section of the spec that lives on the Workflow level but can be overridden on the Artifact level (option 3 from slide). Another option considered was to just allow specification on the Workflow level (option 2 from slide) so as to reduce the complexity of the code and reduce the potential number of Pods running, but Option 3 was selected in the end to maximize flexibility.
-
Summarizing ADR statement:
-"In the context of Artifact Garbage Collection, facing the question of how users should specify Service Account and annotations, we decided to give them the option to specify them on the Workflow level and/or override them on the Artifact level, to maximize flexibility for user needs, accepting that the code will be more complicated, and sometimes there will be many Pods running."
We can reject the Workflow during validation if ArtifactGC is configured along with a non-supported storage engine (for now probably anything besides S3).
Need to clarify certain things in our documentation:
-
-
Users need to know that if they don't name their artifacts with unique keys, they risk the same key being deleted by one Workflow and created by another at the same time. One recommendation is to parametrize the key, e.g. {{workflow.uid}}/hello.txt.
-
Requirement to specify Service Account or Annotation for ArtifactGC specifically if they are needed (we won't fall back to default Workflow SA/annotations). Also, the Service Account needs to either be bound to the "agent" role or otherwise allow the same access to ArtifactGCTasks.
Currently, CronWorkflows are a great resource if we want to run recurring tasks to infinity. However, it is missing the ability to customize it, for example define how many times a workflow should run or how to handle multiple failures. I believe argo workflows would benefit of having more configuration options for cron workflows, to allow to change its behavior based on the result of its child’s success or failures. Below I present my thoughts on how we could improve them, but also some questions and concerns on how to properly do it.
This proposal discusses the viability of adding 2 more fields into the cron workflow configuration:
-
RunStrategy:
-maxSuccess:
-maxFailures:
-
-
maxSuccess - defines how many child workflows must have success before suspending the workflow schedule
-
maxFailures - defines how many child workflows must fail before suspending the workflow scheduling. This may contain Failed workflows, Errored workflows or spec errors.
-
For example, if we want to run a workflow just once, we could just set:
-
RunStrategy:
-maxSuccess:1
-
-
This configuration will make sure the controller will keep scheduling workflows until one of them finishes with success.
-
As another example, if we want to stop scheduling workflows when they keep failing, we could configure the CronWorkflow with:
-
RunStrategy:
-maxFailures:2
-
-
This config will stop scheduling workflows if fails twice.
One aspect that needs to be discussed is whether these configurations apply to the entire life of a cron Workflow or just in consecutive schedules. For example, if we configure a workflow to stop scheduling after 2 failures, I think it makes sense to have this applied when it fails twice consecutively. Otherwise, we can have 2 outages in different periods which will suspend the workflow. On the other hand, when configuring a workflow to run twice with success, it would make more sense to have it execute with success regardless of whether it is a consecutive success or not. If we have an outage after the first workflow succeeds, which translates into failed workflows, it should need to execute with success only once. So I think it would make sense to have:
-
-
-
maxFailures - maximum number of consecutive failures before stopping the scheduling of a workflow
-
-
-
maxSuccess - maximum number of workflows with success.
Since we need to control how many child workflows had success/failure we must store state. With this some questions arise:
-
-
-
Should we just store it through the lifetime of the controller or should we store it to a database?
-
-
Probably only makes sense if we can backup the state somewhere (like a BD). However, I don't have enough knowledge about workflow's architecture to tell how good of an idea this is.
-
-
If a CronWorkflow gets re-applied, does it maintain or reset the number of success/failures?
-
-
-
I guess it should reset since a configuration change should be seen as a new start.
Once the configured number of failures or successes is reached, it is necessary to stop the workflow scheduling.
-I believe we have 3 options:
-
-
Delete the workflow: In my opinion, this is the worst option and goes against gitops principles.
-
Suspend it (set suspend=true): the workflow spec is changed to have the workflow suspended. I may be wrong but this conflicts with gitops as well.
-
Stop scheduling it: The workflow spec is the same. The controller needs to check if the max number of runs was already attained and skip scheduling if it did.
-
-
Option 3 seems to be the only possibility. After reaching the max configured executions, the cron workflow would exist forever but never scheduled. Maybe we could add a new status field, like Inactive and have something the UI to show it?
One possible case that comes to mind is a long outage where all workflows are failing. For example, imagine a workflow that needs to download a file from some storage and for some reason that storage is down. Workflows will keep getting scheduled but they are going to fail. If they fail the number of configured maxFailures, the workflows gets stopped forever. Once the storage is back up, how can the user enable the workflow again?
-
-
Manually re-create the workflow: could be an issue if the user has multiple cron workflows
-
Instead of stopping the workflow scheduling, introduce a back-off period as suggested by #7291. Or maybe allow both configurations.
-
-
I believe option 2 would allow the user to select if they want to stop scheduling or not. If they do, when cron workflows are wrongfully halted, they will need to manually start them again. If they don't, Argo will only introduce a back-off period between schedules to avoid rescheduling workflows that are just going to fail. Spec could look something like:
-
RunStrategy:
-maxSuccess:
-maxFailures:
-value:# this would be optional
-back-off:
-enabled:true
-factor:2
-
-
With this configuration the user could configure 3 behaviors:
-
-
set value if they wanted to stop scheduling a workflow after a maximum number of consecutive failures.
-
set value and back-off if they wanted to stop scheduling a workflow after a maximum number of consecutive failures but with a back-off period between each failure
-
set back-off if they want a back-off period between each failure but they never want to stop the workflow scheduling.
I believe this feature would enhance the cron workflows to allow more specific use cases that are commonly requested by the community, such as running a workflow only once. This proposal raises some concerns on how to properly implement it and I would like to know the maintainers/contributors opinion on these 4 topics, but also some other issues that I couldn't think of.
The motivation for this proposal is to enable developers working on Argo Workflows to use build tools in a more reproducible way.
-Currently the Makefile is unfortunately too opinionated and as a result is often a blocker when first setting up Argo Workflows locally.
-I believe we should shrink the responsibilities of the Makefile and where possible outsource areas of responsibility to more specialized technology, such
-as Devenv/Nix in the case of dependency management.
In order to better address reproducibility, it is better to split up the duties the Makefile currently performs into various sub components, that can be assembled in more appropriate technology. One important aspect here is to completely shift the responsibility of dependency management away from the Makefile and into technology such as Nix or Devenv. This proposal will also enable quicker access to a development build of Argo Workflows to developers, reducing the costs of on-boarding and barrier to entry.
I suggest that we use Nix over Devenv. I believe that our build environment is unique enough that we will be tapping into Nix anyway, it probably makes sense to directly use Nix in that case.
In order to maximize the benefit we receive from using something like Nix, I suggest that we initially start off with a modest change to the Makefile.
-The first proposal would be to remove out all dependency management code and replace this functionality with Nix, where it is trivially possible. This may not be possible for some go lang related binaries we use, we will retain the Makefile functionality in those cases, at least for a while. Eventually we will migrate more and more of this responsibility away from the Makefile. Following Nix being responsible for all dependency management, we could start to consider moving more of our build system itself into Nix, perhaps it is easiest to start off with UI build as it is relatively painless. However, do note that this is not a requirement, I do not see a problem with the Makefile and the Nix file co-existing, it is more about finding a good balance between the reproducibility we desire and the effort we put into obtaining said reproducibility. An example for a replacement could be this dependency for example, note that we do not state any version here, replacing such installations with Nix based installations will ensure that we can ensure that if a build works on a certain developer's machine, it should also work on every other machine as well.
As mentioned previously Nix gets us closer to reproducible build environments. It should ease significantly the on-boarding process of developers onto the project.
-There have been several developers who wanted to work on Argo Workflows but found the Makefile to be a barrier, it is likely that there are more developers on this boat. With a reproducible build environment, we hope that
-everyone who would like to contribute to the project is able to do so easily. It should also save time for engineers on-boarding onto the project, especially if they are using a system that is not Ubuntu or OSX.
If we proceed further with Nix, it will require some amount of people working on Argo Workflows to learn it, this is not a trivial task by any means.
-It will increase the barrier when it comes to changes that are build related, however, this isn't necessarily bad as build related changes should be far less frequent, the friction we will endure here is likely manageable.
In the case that both Nix and the Makefile co-exist, we could use nix inside the Makefile itself. The Makefile calls into Nix to setup a developer environment with all dependencies, it will then continue the rest of the Makefile execution as normal.
-Following a complete or near complete migration to Nix, we can use nix-build for more of our tasks. An example of a C++ project environment is provided here
To see how Argo Workflows work, you can install it and run examples of simple workflows.
-
Before you start you need a Kubernetes cluster and kubectl set up to be able to access that cluster. For the purposes of getting up and running, a local cluster is fine. You could consider the following local Kubernetes cluster options:
Alternatively, if you want to try out Argo Workflows and don't want to set up a Kubernetes cluster, try the Killercoda course.
-
-
Development vs. Production
-
These instructions are intended to help you get started quickly. They are not suitable in production. For production installs, please refer to the installation documentation.
The argo-server (and thus the UI) defaults to client authentication, which requires clients to provide their Kubernetes bearer token in order to authenticate. For more information, refer to the Argo Server Auth Mode documentation. We will switch the authentication mode to server so that we can bypass the UI login for now:
This will serve the UI on https://localhost:2746. Due to the self-signed certificate, you will receive a TLS error which you will need to manually approve.
-
-
Pay close attention to the URI. It uses https and not http. Navigating to http://localhost:2746 result in server-side error that breaks the port-forwarding.
The --watch flag used above will allow you to observe the workflow as it runs and the status of whether it succeeds.
-When the workflow completes, the watch on the workflow will stop.
-
You can list all the Workflows you have submitted by running the command below:
-
argolist-nargo
-
-
You will notice the Workflow name has a hello-world- prefix followed by random characters. These characters are used
-to give Workflows unique names to help identify specific runs of a Workflow. If you submitted this Workflow again,
-the next Workflow run would have a different name.
-
Using the argo get command, you can always review details of a Workflow run. The output for the command below will
-be the same as the information shown as when you submitted the Workflow:
-
argoget-nargo@latest
-
-
The @latest argument to the CLI is a short cut to view the latest Workflow run that was executed.
-
You can also observe the logs of the Workflow run by running the following:
Versions are expressed as x.y.z, where x is the major version, y is the minor version, and z is the patch version,
-following Semantic Versioning terminology.
-
Argo Workflows does not use Semantic Versioning. Minor versions may contain breaking changes. Patch versions only
-contain bug fixes and minor features.
-
For stable, use the latest patch version.
-
⚠️ Read the upgrading guide to find out about breaking changes before any upgrade.
New minor versions are released roughly every 6 months.
-
Release candidates (RCs) for major and minor releases are typically available for 4-6 weeks before the release becomes generally available (GA). Features may be shipped in subsequent release candidates.
-
When features are shipped in a new release candidate, the most recent release candidate will be available for at least 2 weeks to ensure it is tested sufficiently before it is pushed to GA. If bugs are found with a feature and are not resolved within the 2 week period, the features will be rolled back so as to be saved for the next major/minor release timeline, and a new release candidate will be cut for testing before pushing to GA.
-
Otherwise, we typically release every two weeks:
-
-
Patch fixes for the current stable version.
-
The next release candidate, if we are currently in a release-cycle.
Argo versions may be compatible with newer and older versions than what it is listed but only three minor versions are supported per Argo release unless otherwise noted.
-
The main branch of Argo Workflows is currently tested on Kubernetes 1.27.
Then look for "failed to cherry-pick" in the log to find commits that fail to be cherry-picked and decide if a
-manual patch is necessary.
-
Ignore:
-
-
Fixes for features only on main.
-
Dependency upgrades, unless they fix known security issues.
-
Build or CI improvements, unless the release pipeline is blocked without them.
-
-
Cherry-pick the first commit. Run make test locally before pushing. If the build timeouts the build caches may have
-gone, try re-running.
-
Don't cherry-pick another commit until the CI passes. It is harder to find the cause of a new failed build if the last
-build failed too.
-
Cherry-picking commits one-by-one and then waiting for the CI will take a long time. Instead, cherry-pick each commit then
-run make test locally before pushing.
Once the tag is published, GitHub Actions will automatically open a PR to update the changelog. Once the PR is ready,
-you can approve it, enable auto-merge, and then run the following to force trigger the CI build:
-
gitbranch-Dcreate-pull-request/changelog
-gitfetchupstream
-gitcheckout--trackupstream/create-pull-request/changelog
-gitcommit-s--allow-empty-m"docs: Force trigger CI"
-gitpushupstreamcreate-pull-request/changelog
-
Argo Workflows provides an indication of how much resource your workflow has used and saves this
-information. This is intended to be an indicative but not accurate value.
The calculation is always an estimate, and is calculated by duration.go
-based on container duration, specified pod resource requests, limits, or (for memory and CPU)
-defaults.
-
Each indicator is divided by a common denominator depending on resource type.
Each resource type has a denominator used to make large values smaller.
-
-
CPU: 1
-
Memory: 100Mi
-
Storage: 10Gi
-
Ephemeral Storage: 10Gi
-
All others: 1
-
-
The requested fraction of the base amount will be multiplied by the container's run time to get
-the container's Resource Duration.
-
For example, if you've requested 50Mi of memory (half of the base amount), and the container
-runs 120sec, then the reported Resource Duration will be 60sec * (100Mi memory).
Both the web and CLI give abbreviated usage, like 9m10s*cpu,6s*memory,2m31s*nvidia.com/gpu. In
-this context, resources like memory refer to the "base amounts".
-
For example, memory means "amount of time a resource requested 100Mi of memory." If a container only
-uses 10Mi, each second it runs will only count as a tenth-second of memory.
For a short running pods (<10s), if the memory request is also small (for example, 10Mi), then the memory value may be 0s. This is because the denominator is 100Mi.
Argo Workflows ships with a server that provides more features and security than before.
-
The server can be configured with or without client auth (server --auth-mode client). When it is disabled, then clients must pass their KUBECONFIG base 64 encoded in the HTTP Authorization header:
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:retry-container-
-spec:
-entrypoint:retry-container
-templates:
--name:retry-container
-retryStrategy:
-limit:"10"
-container:
-image:python:alpine3.6
-command:["python",-c]
-# fail with a 66% probability
-args:["importrandom;importsys;exit_code=random.choice([0,1,1]);sys.exit(exit_code)"]
-
-
The retryPolicy and expression are re-evaluated after each attempt. For example, if you set retryPolicy: OnFailure and your first attempt produces a failure then a retry will be attempted. If the second attempt produces an error, then another attempt will not be made.
Use retryPolicy to choose which failure types to retry:
-
-
Always: Retry all failed steps
-
OnFailure: Retry steps whose main container is marked as failed in Kubernetes
-
OnError: Retry steps that encounter Argo controller errors, or whose init or wait containers fail
-
OnTransientError: Retry steps that encounter errors defined as transient, or errors matching the TRANSIENT_ERROR_PATTERNenvironment variable. Available in version 3.0 and later.
-
-
The retryPolicy applies even if you also specify an expression, but in version 3.5 or later the default policy means the expression makes the decision unless you explicitly specify a policy.
-
The default retryPolicy is OnFailure, except in version 3.5 or later when an expression is also supplied, when it is Always. This may be easier to understand in this diagram.
-
flowchart LR
- start([Will a retry be attempted])
- start --> policy
- policy(Policy Specified?)
- policy-->|No|expressionNoPolicy
- policy-->|Yes|policyGiven
- policyGiven(Expression Specified?)
- policyGiven-->|No|policyGivenApplies
- policyGiven-->|Yes|policyAndExpression
- policyGivenApplies(Supplied Policy)
- policyAndExpression(Supplied Policy AND Expression)
- expressionNoPolicy(Expression specified?)
- expressionNoPolicy-->|No|onfailureNoExpr
- expressionNoPolicy-->|Yes|version
- onfailureNoExpr[OnFailure]
- onfailure[OnFailure AND Expression]
- version(Workflows version)
- version-->|3.4 or ealier|onfailure
- always[Only Expression matters]
- version-->|3.5 or later|always
-
-
An example retry strategy:
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:retry-on-error-
-spec:
-entrypoint:error-container
-templates:
--name:error-container
-retryStrategy:
-limit:"2"
-retryPolicy:"Always"
-container:
-image:python
-command:["python","-c"]
-# fail with a 80% probability
-args:["importrandom;importsys;exit_code=random.choice(range(0,5));sys.exit(exit_code)"]
-
Argo Workflows is an incredibly scalable tool for orchestrating workflows. It empowers you to process thousands of workflows per day, with each workflow consisting of tens of thousands of nodes. Moreover, it effortlessly handles hundreds of thousands of smaller workflows daily. However, optimizing your setup is crucial to fully leverage this capability.
You must be running at least v3.1 for several recommendations to work. Upgrade to the very latest patch. Performance
-fixes often come in patches.
-
Test Your Cluster Before You Install Argo Workflows¶
-
You'll need a big cluster, with a big Kubernetes master.
-
Users often encounter problems with Kubernetes needing to be configured for the scale. E.g. Kubernetes API server being
-too small. We recommend you test your cluster to make sure it can run the number of pods they need, even before
-installing Argo. Create pods at the rate you expect that it'll be created in production. Make sure Kubernetes can keep
-up with requests to delete pods at the same rate.
-
You'll need to GC data quickly. The less data that Kubernetes and Argo deal with, the less work they need to do. Use
-pod GC and workflow GC to achieve this.
If you're running workflows with many nodes, you'll probably be offloading data to a database. Offloaded data is kept
-for 5m. You can reduce the number of records created by setting DEFAULT_REQUEUE_TIME=1m. This will slow reconciliation,
-but will suit workflows where nodes run for over 1m.
The development container should be able to do everything you need to do to develop Argo Workflows without installing tools on your local machine. It takes quite a long time to build the container. It runs k3d inside the container so you have a cluster to test against. To communicate with services running either in other development containers or directly on the local machine (e.g. a database), the following URL can be used in the workflow spec: host.docker.internal:<PORT>. This facilitates the implementation of workflows which need to connect to a database or an API server.
-
You can use the development container in a few different ways:
-
-
Visual Studio Code with Dev Containers extension. Open your argo-workflows folder in VSCode and it should offer to use the development container automatically. VSCode will allow you to forward ports to allow your external browser to access the running components.
-
devcontainer CLI. Once installed, go to your argo-workflows folder and run devcontainer up --workspace-folder . followed by devcontainer exec --workspace-folder . /bin/bash to get a shell where you can build the code. You can use any editor outside the container to edit code; any changes will be mirrored inside the container. Due to a limitation of the CLI, only port 8080 (the Web UI) will be exposed for you to access if you run this way. Other services are usable from the shell inside.
-
GitHub Codespaces. You can start editing as soon as VSCode is open, though you may want to wait for pre-build.sh to finish installing dependencies, building binaries, and setting up the cluster before running any commands in the terminal. Once you start running services (see next steps below), you can click on the "PORTS" tab in the VSCode terminal to see all forwarded ports. You can open the Web UI in a new tab from there.
We recommend using K3D to set up the local Kubernetes cluster since this will allow you to test RBAC
-set-up and is fast. You can set-up K3D to be part of your default kube config as follows:
-
k3dclusterstart--wait
-
-
Alternatively, you can use Minikube to set up the local Kubernetes cluster.
-Once a local Kubernetes cluster has started via minikube start, your kube config will use Minikube's context
-automatically.
-
-
Warning
-
Do not use Docker Desktop's embedded Kubernetes, it does not support Kubernetes RBAC (i.e. kubectl auth can-i always returns allowed).
In most cases, you want to run the test that relates to your changes locally. You should not run all the tests suites.
-Our CI will run those concurrently when you create a PR, which will give you feedback much faster.
When running make pre-commit -B, if you encounter errors like
- make: *** [pkg/apiclient/clusterworkflowtemplate/cluster-workflow-template.swagger.json] Error 1, ensure that you
- have checked out your code into $GOPATH/src/github.com/argoproj/argo-workflows.
-
If you encounter "out of heap" issues when building UI through Docker, please validate resources allocated to Docker.
- Compilation may fail if allocated RAM is less than 4Gi.
-
To start profiling with pprof, pass ARGO_PPROF=true when starting the controller locally.
- Then run the following:
I run the controller in one terminal, and the UI in another. I like the UI: it is much faster to debug workflows than
-the terminal. This allows you to make changes to the controller and re-start it, without restarting the UI (which I
-think takes too long to start-up).
-
As a convenience, CTRL=false implies UI=true, so just run:
Nix is a package manager / build tool which focuses on reproducible build environments.
-Argo Workflows has some basic support for Nix which is enough to get Argo Workflows up and running with minimal effort.
-Here are the steps to follow:
-
-
Modify your hosts file and set up a Kubernetes cluster according to Running Locally. Don't worry about the other instructions.
Run nix develop --extra-experimental-features nix-command --extra-experimental-features flakes ./dev/nix/ --impure (you can add the extra features as a default in your nix.conf file).
This is still bare-bones at the moment, any feature in the Makefile not mentioned here is excluded for now.
-In practice, this means that only a make start UI=true equivalent is supported at the moment.
-As an additional caveat, there are no LDFlags set in the build; as a result the UI will show 0.0.0-unknown for the version.
Most dependencies are in the Nix packages repository but if you want a specific version, you might have to build it yourself.
-This is fairly trivial in Nix, the idea is to just change the version string to whatever package you are concerned about.
If we look at the mkdocs dependency, we see a call to buildPythonPackage, to change the version we need to just modify the version string.
-Doing this will display a failure because the hash from the fetchPypi command will now differ, it will also display the correct hash, copy this hash
-and replace the existing hash value.
The almost exact same principles apply here, the only difference being you must change the vendorHash and the sha256 fields.
-The vendorHash is a hash of the vendored dependencies while the sha256 is for the sources fetched from the fetchFromGithub call.
Unfortunately, dependabot is not capable of upgrading flakes automatically, when the go modules are automatically upgraded the
-hash of the vendor dependencies changes but this change isn't automatically reflected in the nix file. The vendorSha256 field that needs to
-be upgraded can be found by searching for ${package.name} = pkgs.buildGoModule in the nix file.
The K8S client library rate limits the messages that can go out.
-
If you frequently see messages similar to this in the Controller log (issued by the library):
-
Waited for 7.090296384s due to client-side throttling, not priority and fairness, request: GET:https://10.100.0.1:443/apis/argoproj.io/v1alpha1/namespaces/argo/workflowtemplates/s2t
-
-
Or, in >= v3.5, if you see warnings similar to this (could be any CR, not just WorkflowTemplate):
-
Waited for 7.090296384s, request:GET:https://10.100.0.1:443/apis/argoproj.io/v1alpha1/namespaces/argo/workflowtemplates/s2t
-
-
Then, if your K8S API Server can handle more requests:
-
-
Increase both --qps and --burst arguments for the Controller. The qps value indicates the average number of queries per second allowed by the K8S Client. The burst value is the number of queries/sec the Client receives before it starts enforcing qps, so typically burst > qps. If not set, the default values are qps=20 and burst=30 (as of v3.5 (refer to cmd/workflow-controller/main.go in case the values change)).
The controller has permission (via Kubernetes RBAC + its config map) with either all namespaces (cluster-scope install) or a single managed namespace (namespace-install), notably:
-
-
List/get/update workflows, and cron-workflows.
-
Create/get/delete pods, PVCs, and PDBs.
-
List/get template, config maps, service accounts, and secrets.
Users minimally need permission to create/read workflows. The controller will then create workflow pods (config maps etc) on behalf of the users, even if the user does not have permission to do this themselves. The controller will only create workflow pods in the workflow's namespace.
-
A way to think of this is that, if the user has permission to create a workflow in a namespace, then it is OK to create pods or anything else for them in that namespace.
-
If the user only has permission to create workflows, then they will be typically unable to configure other necessary resources such as config maps, or view the outcome of their workflow. This is useful when the user is a service.
-
-
Warning
-
If you allow users to create workflows in the controller's namespace (typically argo), it may be possible for users to modify the controller itself. In a namespace-install the managed namespace should therefore not be the controller's namespace.
-
-
You can typically further restrict what a user can do to just being able to submit workflows from templates using the workflow restrictions feature.
Different service accounts should be used if a workflow pod needs to have elevated permissions, e.g. to create other resources.
-
The main container will have the service account token mounted, allowing the main container to patch pods (among other permissions). Set automountServiceAccountToken to false to prevent this. See fields.
Argo Workflows requires various levels of network access depending on configuration and the features enabled. The following describes the different workflow components and their network access needs, to help provide guidance on how to configure the argo namespace in a secure manner (e.g. NetworkPolicy).
The Argo Server is commonly exposed to end-users to provide users with a UI for visualizing and managing their workflows. It must also be exposed if leveraging webhooks to trigger workflows. Both of these use cases require that the argo-server Service to be exposed for ingress traffic (e.g. with an Ingress object or load balancer). Note that the Argo UI is also available to be accessed by running the server locally (i.e. argo server) using local KUBECONFIG credentials, and visiting the UI over https://localhost:2746.
-
The Argo Server additionally has a feature to allow downloading of artifacts through the UI. This feature requires that the argo-server be given egress access to the underlying artifact provider (e.g. S3, GCS, MinIO, Artifactory, Azure Blob Storage) in order to download and stream the artifact.
The workflow-controller Deployment exposes a Prometheus metrics endpoint (workflow-controller-metrics:9090) so that a Prometheus server can periodically scrape for controller level metrics. Since Prometheus is typically running in a separate namespace, the argo namespace should be configured to allow cross-namespace ingress access to the workflow-controller-metrics Service.
A persistent store can be configured for either archiving or offloading workflows. If either of these features are enabled, both the workflow-controller and argo-server Deployments will need egress network access to the external database used for archiving/offloading.
In order for Argo to support features such as artifacts, outputs, access to secrets, etc. it needs to communicate with Kubernetes resources
-using the Kubernetes API. To communicate with the Kubernetes API, Argo uses a ServiceAccount to authenticate itself to the Kubernetes API.
-You can specify which Role (i.e. which permissions) the ServiceAccount that Argo uses by binding a Role to a ServiceAccount using a RoleBinding
-
Then, when submitting Workflows you can specify which ServiceAccount Argo uses using:
-
argosubmit--serviceaccount<name>
-
-
When no ServiceAccount is provided, Argo will use the defaultServiceAccount from the namespace from which it is run, which will almost always have insufficient privileges by default.
-
For more information about granting Argo the necessary permissions for your use case see Workflow RBAC.
For the purposes of this demo, we will grant the defaultServiceAccount admin privileges (i.e., we will bind the adminRole to the defaultServiceAccount of the current namespace):
Note that this will grant admin privileges to the defaultServiceAccount in the namespace that the command is run from, so you will only be able to
-run Workflows in the namespace where the RoleBinding was made.
Automatic (i.e. mutating webhook based) sidecar injection systems, including service meshes such as Anthos and Istio
-Proxy, create a unique problem for Kubernetes workloads that run to completion.
-
Because sidecars are injected outside of the view of the workflow controller, the controller has no awareness of them.
-It has no opportunity to rewrite the containers command (when using the Emissary Executor) and as the sidecar's process
-will run as PID 1, which is protected. It can be impossible for the wait container to terminate the sidecar.
-
You will minimize problems by not using Istio with Argo Workflows.
Kubernetes does not provide a way to kill a single container. You can delete a pod, but this kills all containers, and loses all information
-and logs of that pod.
-
Instead, try to mimic the Kubernetes termination behavior, which is:
-
-
SIGTERM PID 1
-
Wait for the pod's terminateGracePeriodSeconds (30s by default).
-
SIGKILL PID 1
-
-
The following are not supported:
-
-
preStop
-
STOPSIGNAL
-
-
To do this, it must be possible to run a kubectl exec command that kills the injected sidecar. By default it runs /bin/sh -c 'kill 1'. This can fail:
-
-
No /bin/sh.
-
Process is not running as PID 1 (which is becoming the default these days due to runAsNonRoot).
-
Process does not correctly respond to kill 1 (e.g. some shell script weirdness).
-
-
You can override the kill command by using a pod annotation (where %d is the signal number), for example:
# Login to GCP:
-gloudauthlogin
-
-# Set-up your config (if needed):
-gcloudconfigsetprojectalex-sb
-
-# Create a cluster (default region is us-west-2, if you're not in west of the USA, you might want at different region):
-gcloudcontainerclusterscreate-autoargo-workflows-stress-1
-
-# Get credentials:
-gcloudcontainerclustersget-credentialsargo-workflows-stress-1
-
-# Install workflows (If this fails, try running it again):
-makestartPROFILE=stress
-
-# Make sure pods are running:
-kubectlgetdeployments
-
-# Run a test workflow:
-argosubmitexamples/hello-world.yaml--watch
-
Run go run ./test/stress/tool -n 10000 to run a large number of workflows.
-
Check Prometheus:
-
-
See how many Kubernetes API requests are being made. You will see about one Update workflows
- per reconciliation, multiple Create pods. You should expect to see one Get workflowtemplates per workflow (done
- on first reconciliation). Otherwise, if you see anything else, that might be a problem.
-
How many errors were logged? log_messages{level="error"} What was the cause?
Synchronization enables users to limit the parallel execution of certain workflows or
-templates within a workflow without having to restrict others.
-
Users can create multiple synchronization configurations in the ConfigMap that can be referred to
-from a workflow or template within a workflow. Alternatively, users can
-configure a mutex to prevent concurrent execution of templates or
-workflows using the same mutex.
-
For example:
-
apiVersion:v1
-kind:ConfigMap
-metadata:
-name:my-config
-data:
-workflow:"1"# Only one workflow can run at given time in particular namespace
-template:"2"# Two instances of template can run at a given time in particular namespace
-
Workflow-level synchronization limits parallel execution of the workflow if workflows have the same synchronization reference.
-In this example, Workflow refers to workflow synchronization key which is configured as limit 1,
-so only one workflow instance will be executed at given time even multiple workflows created.
Template-level synchronization limits parallel execution of the template across workflows, if templates have the same synchronization reference.
-In this example, acquire-lock template has synchronization reference of template key which is configured as limit 2,
-so two instances of templates will be executed at a given time: even multiple steps/tasks within workflow or different workflows referring to the same template.
In addition to this synchronization, the workflow controller supports a parallelism setting that applies to all workflows
-in the system (it is not granular to a class of workflows, or tasks withing them). Furthermore, there is a parallelism setting
-at the workflow and template level, but this only restricts total concurrent executions of tasks within the same workflow.
TemplateDefaults feature enables the user to configure the default template values in workflow spec level that will apply to all the templates in the workflow. If the template has a value that also has a default value in templateDefault, the Template's value will take precedence. These values will be applied during the runtime. Template values and default values are merged using Kubernetes strategic merge patch. To check whether and how list values are merged, inspect the patchStrategy and patchMergeKey tags in the workflow definition.
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-name:template-defaults-example
-spec:
-entrypoint:main
-templateDefaults:
-timeout:30s# timeout value will be applied to all templates
-retryStrategy:# retryStrategy value will be applied to all templates
-limit:2
-templates:
--name:main
-container:
-image:docker/whalesay:latest
-
Operator can configure the templateDefaults in workflow defaults. This templateDefault will be applied to all the workflow which runs on the controller.
-
The following would be specified in the Config Map:
-
apiVersion:v1
-kind:ConfigMap
-metadata:
-name:workflow-controller-configmap
-data:
-# Default values that will apply to all Workflows from this controller, unless overridden on the Workflow-level
-workflowDefaults:|
-metadata:
-annotations:
-argo: workflows
-labels:
-foo: bar
-spec:
-ttlStrategy:
-secondsAfterSuccess: 5
-templateDefaults:
-timeout: 30s
-
In Kubernetes, pods are cattle and can be deleted at any time. Deletion could be manually via kubectl delete pod, during a node drain, or for other reasons.
-
This can be very inconvenient, your workflow will error, but for reasons outside of your control.
-
A pod disruption budget can reduce the likelihood of this happening. But, it cannot entirely prevent it.
-
To retry pods that were deleted, set retryStrategy.retryPolicy: OnError.
-
This can be set at a workflow-level, template-level, or globally (using workflow defaults)
We've created a Killercoda course featuring beginner and intermediate lessons. These allow to you try out Argo Workflows in your web browser without needing to install anything on your computer. Each lesson starts up a Kubernetes cluster that you can access via a web browser.
Emissary executor is now the only supported executor. If you are using other executors, e.g. docker, k8sapi, pns, and kubelet, you need to
-remove your containerRuntimeExecutors and containerRuntimeExecutor from your controller's configmap. If you have workflows that use different
-executors with the label workflows.argoproj.io/container-runtime-executor, this is no longer supported and will not be effective.
-
chore!: Remove dataflow pipelines from codebase. (#9071)¶
-
You are affected if you are using dataflow pipelines in the UI or via the /pipelines endpoint.
-We no longer support dataflow pipelines and all relevant code has been removed.
This PR automatically looks up the command and entrypoint. The implementation for config look-up was incorrect (it
-allowed you to specify args but not entrypoint). args has been removed to correct the behaviour.
-
If you are incorrectly configured, the workflow controller will error on start-up.
For AWS users using S3: visualizing artifacts in the UI and downloading them now requires an additional "Action" to be configured in your S3 bucket policy: "ListBucket".
The PR changes the permissions used by the agent to report back the outcome of HTTP template requests. The permission patch workflowtasksets/status replaces patch workflowtasksets, for example:
executorImage (use executor.image in configmap instead)
- e.g.
- Workflow controller configmap similar to the following one given below won't be valid anymore:
executorImagePullPolicy (use executor.imagePullPolicy in configmap instead)
- e.g.
- Workflow controller configmap similar to the following one given below won't be valid anymore:
executorResources (use executor.resources in configmap instead)
- e.g.
- Workflow controller configmap similar to the following one given below won't be valid anymore:
This PR removes pod workers from the code, the pod informer directly writes into the workflow queue. As a result the --pod-workers flag has been removed.
-
93c11a24ff feat: Add TLS to Metrics and Telemetry servers (#7041)¶
-
This PR adds the ability to send metrics over TLS with a self-signed certificate. In v3.5 this will be enabled by default, so it is recommended that users enable this functionality now.
-
0758eab11 feat(server)!: Sync dispatch of webhook events by default¶
-
This is not expected to impact users.
-
Events dispatch in the Argo Server has been change from async to sync by default. This is so that errors are surfaced to
-the client, rather than only appearing as logs or Kubernetes events. It is possible that response times under load are
-too long for your client and you may prefer to revert this behaviour.
-
To revert this behaviour, restart Argo Server with ARGO_EVENT_ASYNC_DISPATCH=true. Make sure that asyncDispatch=true
-is logged.
-
bd49c6303 fix(artifact)!: default https to any URL missing a scheme. Fixes #6973¶
-
HTTPArtifact without a scheme will now defaults to https instead of http
-
user need to explicitly include a http prefix if they want to retrieve HTTPArtifact through http
-
chore!: Remove the hidden flag --verify from argo submit¶
-
The hidden flag --verify has been removed from argo submit. This is a internal testing flag we don't need anymore.
e5b131a33 feat: Add template node to pod name. Fixes #1319 (#6712)¶
-
This add the template name to the pod name, to make it easier to understand which pod ran which step. This behaviour can be reverted by setting POD_NAMES=v1 on the workflow controller.
-
be63efe89 feat(executor)!: Change argoexec base image to alpine. Closes #5720 (#6006)¶
-
Changing from Debian to Alpine reduces the size of the argoexec image, resulting is faster starting workflow pods, and it also reduce the risk of security issues. There is not such thing as a free lunch. There maybe other behaviour changes we don't know of yet.
-
Some users found this change prevented workflow with very large parameters from running. See #7586
When upgrading from <v2.12 to >v3.2 workflows that are running at the time of the upgrade and have onExit steps may experience the onExit step running twice. This is only applicable for workflows that began running before a workflow-controller upgrade and are still running after the upgrade is complete. This is only applicable for upgrading from v2.12 or earlier directly to v3.2 or later. Even under these conditions, duplicate work may not be experienced.
The Emissary executor is not a breaking change per-se, but it is brand new so we would not recommend you use it by default yet. Instead, we recommend you test it out on some workflows using a workflow-controller-configmap configuration.
-
# Specifies the executor to use.
-#
-# You can use this to:
-# * Tailor your executor based on your preference for security or performance.
-# * Test out an executor without committing yourself to use it for every workflow.
-#
-# To find out which executor was actually use, see the `wait` container logs.
-#
-# The list is in order of precedence; the first matching executor is used.
-# This has precedence over `containerRuntimeExecutor`.
-containerRuntimeExecutors:|
-- name: emissary
-selector:
-matchLabels:
-workflows.argoproj.io/container-runtime-executor: emissary
-
This PR introduced a new expression syntax know as "expression tag template". A user has reported that this does not
-always play nicely with the when condition syntax (Goevaluate).
-
This can be resolved using a single quote in your when expression:
This change is not breaking per-se, but many users do not appear to aware of artifact repository ref, so check your usage of that feature if you have problems.
You can use HTTP artifacts to connect to webHDFS, where the URL will be the webHDFS endpoint including the file path and any query parameters.
-Suppose your webHDFS endpoint is available under https://mywebhdfsprovider.com/webhdfs/v1/ and you have a file my-art.txt located in a data folder, which you want to use as an input artifact. To construct the URL, you append the file path to the base webHDFS endpoint and set the OPEN operation via query parameter. The result is: https://mywebhdfsprovider.com/webhdfs/v1/data/my-art.txt?op=OPEN.
-See the below Workflow which will download the specified webHDFS artifact into the specified path:
To declare a webHDFS output artifact, instead use the CREATE operation and set the file path to your desired location.
-In the below example, the artifact will be stored at outputs/newfile.txt. You can overwrite existing files with overwrite=true.
The above examples show minimal use cases without authentication. However, in a real-world scenario, you may want to use authentication.
-The authentication mechanism is limited to those supported by HTTP artifacts:
-
-
HTTP Basic Auth
-
OAuth2
-
Client Certificates
-
-
Examples for the latter two mechanisms can be found in the full webHDFS example.
-
-
Provider dependent
-
While your webHDFS provider may support the above mechanisms, Hadoop itself only supports authentication via Kerberos SPNEGO and Hadoop delegation token. HTTP artifacts do not currently support SPNEGO, but delegation tokens can be used via the delegation query parameter.
The tag is substituted with the variable that has a name the same as the tag.
-
Simple tags may have white-space between the brackets and variable as seen below. However, there is a known issue where variables may fail to interpolate with white-space, so it is recommended to avoid using white-space until this issue is resolved. Please report unexpected behavior with reproducible examples.
The tag is substituted with the result of evaluating the tag as an expression.
-
Note that any hyphenated parameter names or step names will cause a parsing error. You can reference them by
-indexing into the parameter or step map, e.g. inputs.parameters['my-param'] or steps['my-step'].outputs.result.
Sprig functions often do not raise errors.
-For example, if int is used on an invalid value, it returns 0.
-Please review the Sprig documentation to understand which functions raise errors and which do not.
Exit code of any previous script or container step
-
-
-
steps.<STEPNAME>.startedAt
-
Time-stamp when the step started
-
-
-
steps.<STEPNAME>.finishedAt
-
Time-stamp when the step finished
-
-
-
steps.<TASKNAME>.hostNodeName
-
Host node where task ran (available from version 3.5)
-
-
-
steps.<STEPNAME>.outputs.result
-
Output result of any previous container or script step
-
-
-
steps.<STEPNAME>.outputs.parameters
-
When the previous step uses withItems or withParams, this contains a JSON array of the output parameter maps of each invocation
-
-
-
steps.<STEPNAME>.outputs.parameters.<NAME>
-
Output parameter of any previous step. When the previous step uses withItems or withParams, this contains a JSON array of the output parameter values of each invocation
Exit code of any previous script or container task
-
-
-
tasks.<TASKNAME>.startedAt
-
Time-stamp when the task started
-
-
-
tasks.<TASKNAME>.finishedAt
-
Time-stamp when the task finished
-
-
-
tasks.<TASKNAME>.hostNodeName
-
Host node where task ran (available from version 3.5)
-
-
-
tasks.<TASKNAME>.outputs.result
-
Output result of any previous container or script task
-
-
-
tasks.<TASKNAME>.outputs.parameters
-
When the previous task uses withItems or withParams, this contains a JSON array of the output parameter maps of each invocation
-
-
-
tasks.<TASKNAME>.outputs.parameters.<NAME>
-
Output parameter of any previous task. When the previous task uses withItems or withParams, this contains a JSON array of the output parameter values of each invocation
When using the expression field within retryStrategy, special variables are available.
-
-
-
-
Variable
-
Description
-
-
-
-
-
lastRetry.exitCode
-
Exit code of the last retry
-
-
-
lastRetry.status
-
Status of the last retry
-
-
-
lastRetry.duration
-
Duration in seconds of the last retry
-
-
-
lastRetry.message
-
Message output from the last retry (available from version 3.5)
-
-
-
-
Note: These variables evaluate to a string type. If using advanced expressions, either cast them to int values (expression: "{{=asInt(lastRetry.exitCode) >= 2}}") or compare them to string values (expression: "{{=lastRetry.exitCode != '2'}}").
When emitting custom metrics in a template, special variables are available that allow self-reference to the current
-step.
-
-
-
-
Variable
-
Description
-
-
-
-
-
status
-
Phase status of the metric-emitting template
-
-
-
duration
-
Duration of the metric-emitting template in seconds (only applicable in Template-level metrics, for Workflow-level use workflow.duration)
-
-
-
exitCode
-
Exit code of the metric-emitting template
-
-
-
inputs.parameters.<NAME>
-
Input parameter of the metric-emitting template
-
-
-
outputs.parameters.<NAME>
-
Output parameter of the metric-emitting template
-
-
-
outputs.result
-
Output result of the metric-emitting template
-
-
-
resourcesDuration.{cpu,memory}
-
Resources duration in seconds. Must be one of resourcesDuration.cpu or resourcesDuration.memory, if available. For more info, see the Resource Duration doc.
Some variables can be emitted in real-time (as opposed to just when the step/task completes). To emit these variables in
-real time, set realtime: true under gauge (note: only Gauge metrics allow for real time variable emission). Metrics
-currently available for real time emission:
Workflow UID. Useful for setting ownership reference to a resource, or a unique artifact location
-
-
-
workflow.parameters.<NAME>
-
Input parameter to the workflow
-
-
-
workflow.parameters
-
All input parameters to the workflow as a JSON string (this is deprecated in favor of workflow.parameters.json as this doesn't work with expression tags and that does)
-
-
-
workflow.parameters.json
-
All input parameters to the workflow as a JSON string
-
-
-
workflow.outputs.parameters.<NAME>
-
Global parameter in the workflow
-
-
-
workflow.outputs.artifacts.<NAME>
-
Global artifact in the workflow
-
-
-
workflow.annotations.<NAME>
-
Workflow annotations
-
-
-
workflow.annotations.json
-
all Workflow annotations as a JSON string
-
-
-
workflow.labels.<NAME>
-
Workflow labels
-
-
-
workflow.labels.json
-
all Workflow labels as a JSON string
-
-
-
workflow.creationTimestamp
-
Workflow creation time-stamp formatted in RFC 3339 (e.g. 2018-08-23T05:42:49Z)
-
-
-
workflow.creationTimestamp.<STRFTIMECHAR>
-
Creation time-stamp formatted with a strftime format character.
-
-
-
workflow.creationTimestamp.RFC3339
-
Creation time-stamp formatted with in RFC 3339.
-
-
-
workflow.priority
-
Workflow priority
-
-
-
workflow.duration
-
Workflow duration estimate in seconds, may differ from actual duration by a couple of seconds
-
-
-
workflow.scheduledTime
-
Scheduled runtime formatted in RFC 3339 (only available for CronWorkflow)
A list of JSON objects containing information about nodes that failed or errored during execution. Available fields: displayName, message, templateName, phase, podName, and finishedAt.
The idea with creating a WorkflowTemplate is that they are reusable bits of code you will use in many actual Workflows. Sometimes it is useful to know which workflow you are part of.
-
workflow.mainEntrypoint is one way you can do this. If each of your actual workflows has a differing entrypoint, you can identify the workflow you're part of. Given this use in a WorkflowTemplate:
This shouldn't be that helpful in logging, you should be able to identify workflows through other labels in your cluster's log tool, but can be helpful when generating metrics for the workflow for example.
In case you want to follow along with this walk-through, here's a quick overview of the most useful argo command line interface (CLI) commands.
-
argosubmithello-world.yaml# submit a workflow spec to Kubernetes
-argolist# list current workflows
-argogethello-world-xxx# get info about a specific workflow
-argologshello-world-xxx# print the logs from a workflow
-argodeletehello-world-xxx# delete workflow
-
-
You can also run workflow specs directly using kubectl, but the Argo CLI provides syntax checking, nicer output, and requires less typing.
When running workflows, it is very common to have steps that generate or consume artifacts. Often, the output artifacts of one step may be used as input artifacts to a subsequent step.
-
The below workflow spec consists of two steps that run in sequence. The first step named generate-artifact will generate an artifact using the whalesay template that will be consumed by the second step named print-message that then consumes the generated artifact.
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:artifact-passing-
-spec:
-entrypoint:artifact-example
-templates:
--name:artifact-example
-steps:
---name:generate-artifact
-template:whalesay
---name:consume-artifact
-template:print-message
-arguments:
-artifacts:
-# bind message to the hello-art artifact
-# generated by the generate-artifact step
--name:message
-from:"{{steps.generate-artifact.outputs.artifacts.hello-art}}"
-
--name:whalesay
-container:
-image:docker/whalesay:latest
-command:[sh,-c]
-args:["cowsayhelloworld|tee/tmp/hello_world.txt"]
-outputs:
-artifacts:
-# generate hello-art artifact from /tmp/hello_world.txt
-# artifacts can be directories as well as files
--name:hello-art
-path:/tmp/hello_world.txt
-
--name:print-message
-inputs:
-artifacts:
-# unpack the message input artifact
-# and put it at /tmp/message
--name:message
-path:/tmp/message
-container:
-image:alpine:latest
-command:[sh,-c]
-args:["cat/tmp/message"]
-
-
The whalesay template uses the cowsay command to generate a file named /tmp/hello-world.txt. It then outputs this file as an artifact named hello-art. In general, the artifact's path may be a directory rather than just a file. The print-message template takes an input artifact named message, unpacks it at the path named /tmp/message and then prints the contents of /tmp/message using the cat command.
-The artifact-example template passes the hello-art artifact generated as an output of the generate-artifact step as the message input artifact to the print-message step. DAG templates use the tasks prefix to refer to another task, for example {{tasks.generate-artifact.outputs.artifacts.hello-art}}.
-
Optionally, for large artifacts, you can set podSpecPatch in the workflow spec to increase the resource request for the init container and avoid any Out of memory issues.
-
<... snipped ...>
-- name:large-artifact
-# below patch gets merged with the actual pod spec and increses the memory
-# request of the init container.
-podSpecPatch:|
-initContainers:
-- name: init
-resources:
-requests:
-memory: 2Gi
-cpu: 300m
-inputs:
-artifacts:
--name:data
-path:/tmp/large-file
-container:
-image:alpine:latest
-command:[sh,-c]
-args:["cat/tmp/large-file"]
-<... snipped ...>
-
-
Artifacts are packaged as Tarballs and gzipped by default. You may customize this behavior by specifying an archive strategy, using the archive field. For example:
-
<... snipped ...>
-outputs:
-artifacts:
-# default behavior - tar+gzip default compression.
--name:hello-art-1
-path:/tmp/hello_world.txt
-
-# disable archiving entirely - upload the file / directory as is.
-# this is useful when the container layout matches the desired target repository layout.
--name:hello-art-2
-path:/tmp/hello_world.txt
-archive:
-none:{}
-
-# customize the compression behavior (disabling it here).
-# this is useful for files with varying compression benefits,
-# e.g. disabling compression for a cached build workspace and large binaries,
-# or increasing compression for "perfect" textual data - like a json/xml export of a large database.
--name:hello-art-3
-path:/tmp/hello_world.txt
-archive:
-tar:
-# no compression (also accepts the standard gzip 1 to 9 values)
-compressionLevel:0
-<... snipped ...>
-
As of version 3.4 you can configure your Workflow to automatically delete Artifacts that you don't need (visit artifact repository capability for the current supported store engine).
-
Artifacts can be deleted OnWorkflowCompletion or OnWorkflowDeletion. You can specify your Garbage Collection strategy on both the Workflow level and the Artifact level, so for example, you may have temporary artifacts that can be deleted right away but a final output that should be persisted:
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:artifact-gc-
-spec:
-entrypoint:main
-artifactGC:
-strategy:OnWorkflowDeletion# default Strategy set here applies to all Artifacts by default
-templates:
--name:main
-container:
-image:argoproj/argosay:v2
-command:
--sh
---c
-args:
--|
-echo "can throw this away" > /tmp/temporary-artifact.txt
-echo "keep this" > /tmp/keep-this.txt
-outputs:
-artifacts:
--name:temporary-artifact
-path:/tmp/temporary-artifact.txt
-s3:
-key:temporary-artifact.txt
--name:keep-this
-path:/tmp/keep-this.txt
-s3:
-key:keep-this.txt
-artifactGC:
-strategy:Never# optional override for an Artifact
-
Consider parameterizing your S3 keys by {{workflow.uid}}, etc (as shown in the example above) if there's a possibility that you could have concurrent Workflows of the same spec. This would be to avoid a scenario in which the artifact from one Workflow is being deleted while the same S3 key is being generated for a different Workflow.
Does your S3 bucket require you to run with a special Service Account or IAM Role Annotation? You can either use the same ones you use for creating artifacts or generate new ones that are specific for deletion permission. Generally users will probably just have a single Service Account or IAM Role to apply to all artifacts for the Workflow, but you can also customize on the artifact level if you need that:
If you do supply your own Service Account you will need to create a RoleBinding that binds it with a role like this:
-
apiVersion:rbac.authorization.k8s.io/v1
-kind:Role
-metadata:
-annotations:
-workflows.argoproj.io/description:|
-This is the minimum recommended permissions needed if you want to use artifact GC.
-name:artifactgc
-rules:
--apiGroups:
--argoproj.io
-resources:
--workflowartifactgctasks
-verbs:
--list
--watch
--apiGroups:
--argoproj.io
-resources:
--workflowartifactgctasks/status
-verbs:
--patch
-
-
This is the artifactgc role if you installed using one of the quick-start manifest files. If you installed with the install.yaml file for the release then the same permissions are in the argo-cluster-role.
-
If you don't use your own ServiceAccount and are just using default ServiceAccount, then the role needs a RoleBinding or ClusterRoleBinding to default ServiceAccount.
If deletion of the artifact fails for some reason (other than the Artifact already having been deleted which is not considered a failure), the Workflow's Status will be marked with a new Condition to indicate "Artifact GC Failure", a Kubernetes Event will be issued, and the Argo Server UI will also indicate the failure. For additional debugging, the user should find 1 or more Pods named <wfName>-artgc-* and can view the logs.
-
If the user needs to delete the Workflow and its child CRD objects, they will need to patch the Workflow to remove the finalizer preventing the deletion:
We also support conditional execution. The syntax is implemented by govaluate which offers the support for complex syntax. See in the example:
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:coinflip-
-spec:
-entrypoint:coinflip
-templates:
--name:coinflip
-steps:
-# flip a coin
---name:flip-coin
-template:flip-coin
-# evaluate the result in parallel
---name:heads
-template:heads# call heads template if "heads"
-when:"{{steps.flip-coin.outputs.result}}==heads"
--name:tails
-template:tails# call tails template if "tails"
-when:"{{steps.flip-coin.outputs.result}}==tails"
---name:flip-again
-template:flip-coin
---name:complex-condition
-template:heads-tails-or-twice-tails
-# call heads template if first flip was "heads" and second was "tails" OR both were "tails"
-when:>-
-( {{steps.flip-coin.outputs.result}} == heads &&
-{{steps.flip-again.outputs.result}} == tails
-) ||
-( {{steps.flip-coin.outputs.result}} == tails &&
-{{steps.flip-again.outputs.result}} == tails )
--name:heads-regex
-template:heads# call heads template if ~ "hea"
-when:"{{steps.flip-again.outputs.result}}=~hea"
--name:tails-regex
-template:tails# call heads template if ~ "tai"
-when:"{{steps.flip-again.outputs.result}}=~tai"
-
-# Return heads or tails based on a random number
--name:flip-coin
-script:
-image:python:alpine3.6
-command:[python]
-source:|
-import random
-result = "heads" if random.randint(0,1) == 0 else "tails"
-print(result)
-
--name:heads
-container:
-image:alpine:3.6
-command:[sh,-c]
-args:["echo\"itwasheads\""]
-
--name:tails
-container:
-image:alpine:3.6
-command:[sh,-c]
-args:["echo\"itwastails\""]
-
--name:heads-tails-or-twice-tails
-container:
-image:alpine:3.6
-command:[sh,-c]
-args:["echo\"itwasheadsthefirstflipandtailsthesecond.Oritwastwotimestails.\""]
-
-
-
Nested Quotes
-
If the parameter value contains quotes, it may invalidate the govaluate expression.
-To handle parameters with quotes, embed an expr expression in the conditional.
-For example:
In this example, we can see how we can use the other template language variable reference (E.g: Jinja) in Argo workflow template.
-Argo will validate and resolve only the variable that starts with an Argo allowed prefix
-{"item", "steps", "inputs", "outputs", "workflow", "tasks"}
Argo workflows can start containers that run in the background (also known as daemon containers) while the workflow itself continues execution. Note that the daemons will be automatically destroyed when the workflow exits the template scope in which the daemon was invoked. Daemon containers are useful for starting up services to be tested or to be used in testing (e.g., fixtures). We also find it very useful when running large simulations to spin up a database as a daemon for collecting and organizing the results. The big advantage of daemons compared with sidecars is that their existence can persist across multiple steps or even the entire workflow.
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:daemon-step-
-spec:
-entrypoint:daemon-example
-templates:
--name:daemon-example
-steps:
---name:influx
-template:influxdb# start an influxdb as a daemon (see the influxdb template spec below)
-
---name:init-database# initialize influxdb
-template:influxdb-client
-arguments:
-parameters:
--name:cmd
-value:curl -XPOST 'http://{{steps.influx.ip}}:8086/query' --data-urlencode "q=CREATE DATABASE mydb"
-
---name:producer-1# add entries to influxdb
-template:influxdb-client
-arguments:
-parameters:
--name:cmd
-value:for i in $(seq 1 20); do curl -XPOST 'http://{{steps.influx.ip}}:8086/write?db=mydb' -d "cpu,host=server01,region=uswest load=$i" ; sleep .5 ; done
--name:producer-2# add entries to influxdb
-template:influxdb-client
-arguments:
-parameters:
--name:cmd
-value:for i in $(seq 1 20); do curl -XPOST 'http://{{steps.influx.ip}}:8086/write?db=mydb' -d "cpu,host=server02,region=uswest load=$((RANDOM % 100))" ; sleep .5 ; done
--name:producer-3# add entries to influxdb
-template:influxdb-client
-arguments:
-parameters:
--name:cmd
-value:curl -XPOST 'http://{{steps.influx.ip}}:8086/write?db=mydb' -d 'cpu,host=server03,region=useast load=15.4'
-
---name:consumer# consume intries from influxdb
-template:influxdb-client
-arguments:
-parameters:
--name:cmd
-value:curl --silent -G http://{{steps.influx.ip}}:8086/query?pretty=true --data-urlencode "db=mydb" --data-urlencode "q=SELECT * FROM cpu"
-
--name:influxdb
-daemon:true# start influxdb as a daemon
-retryStrategy:
-limit:10# retry container if it fails
-container:
-image:influxdb:1.2
-command:
--influxd
-readinessProbe:# wait for readinessProbe to succeed
-httpGet:
-path:/ping
-port:8086
-
--name:influxdb-client
-inputs:
-parameters:
--name:cmd
-container:
-image:appropriate/curl:latest
-command:["/bin/sh","-c"]
-args:["{{inputs.parameters.cmd}}"]
-resources:
-requests:
-memory:32Mi
-cpu:100m
-
-
Step templates use the steps prefix to refer to another step: for example {{steps.influx.ip}}. In DAG templates, the tasks prefix is used instead: for example {{tasks.influx.ip}}.
As an alternative to specifying sequences of steps, you can define a workflow as a directed-acyclic graph (DAG) by specifying the dependencies of each task.
-DAGs can be simpler to maintain for complex workflows and allow for maximum parallelism when running tasks.
-
In the following workflow, step A runs first, as it has no dependencies.
-Once A has finished, steps B and C run in parallel.
-Finally, once B and C have completed, step D runs.
The dependency graph may have multiple roots.
-The templates called from a DAG or steps template can themselves be DAG or steps templates, allowing complex workflows to be split into manageable pieces.
By default, DAGs fail fast: when one task fails, no new tasks will be scheduled.
-Once all running tasks are completed, the DAG will be marked as failed.
-
If failFast is set to false for a DAG, all branches will run to completion, regardless of failures in other branches.
An application of sidecars is to implement Docker-in-Docker (DIND). DIND is useful when you want to run Docker commands from inside a container. For example, you may want to build and push a container image from inside your build container. In the following example, we use the docker:dind image to run a Docker daemon in a sidecar and give the main container access to the daemon.
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:sidecar-dind-
-spec:
-entrypoint:dind-sidecar-example
-templates:
--name:dind-sidecar-example
-container:
-image:docker:19.03.13
-command:[sh,-c]
-args:["untildockerps;dosleep3;done;dockerrun--rmdebian:latestcat/etc/os-release"]
-env:
--name:DOCKER_HOST# the docker daemon can be access on the standard port on localhost
-value:127.0.0.1
-sidecars:
--name:dind
-image:docker:19.03.13-dind# Docker already provides an image for running a Docker daemon
-command:[dockerd-entrypoint.sh]
-env:
--name:DOCKER_TLS_CERTDIR# Docker TLS env config
-value:""
-securityContext:
-privileged:true# the Docker daemon can only run in a privileged container
-# mirrorVolumeMounts will mount the same volumes specified in the main container
-# to the sidecar (including artifacts), at the same mountPaths. This enables
-# dind daemon to (partially) see the same filesystem as the main container in
-# order to use features such as docker volume binding.
-mirrorVolumeMounts:true
-
An exit handler is a template that always executes, irrespective of success or failure, at the end of the workflow.
-
Some common use cases of exit handlers are:
-
-
cleaning up after a workflow runs
-
sending notifications of workflow status (e.g., e-mail/Slack)
-
posting the pass/fail status to a web-hook result (e.g. GitHub build result)
-
resubmitting or submitting another workflow
-
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:exit-handlers-
-spec:
-entrypoint:intentional-fail
-onExit:exit-handler# invoke exit-handler template at end of the workflow
-templates:
-# primary workflow template
--name:intentional-fail
-container:
-image:alpine:latest
-command:[sh,-c]
-args:["echointentionalfailure;exit1"]
-
-# Exit handler templates
-# After the completion of the entrypoint template, the status of the
-# workflow is made available in the global variable {{workflow.status}}.
-# {{workflow.status}} will be one of: Succeeded, Failed, Error
--name:exit-handler
-steps:
---name:notify
-template:send-email
--name:celebrate
-template:celebrate
-when:"{{workflow.status}}==Succeeded"
--name:cry
-template:cry
-when:"{{workflow.status}}!=Succeeded"
--name:send-email
-container:
-image:alpine:latest
-command:[sh,-c]
-args:["echosende-mail:{{workflow.name}}{{workflow.status}}{{workflow.duration}}"]
--name:celebrate
-container:
-image:alpine:latest
-command:[sh,-c]
-args:["echohooray!"]
--name:cry
-container:
-image:alpine:latest
-command:[sh,-c]
-args:["echoboohoo!"]
-
You can use any container image to generate any kind of artifact. In practice, however, certain types of artifacts are very common, so there is built-in support for git, HTTP, GCS, and S3 artifacts.
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:hardwired-artifact-
-spec:
-entrypoint:hardwired-artifact
-templates:
--name:hardwired-artifact
-inputs:
-artifacts:
-# Check out the main branch of the argo repo and place it at /src
-# revision can be anything that git checkout accepts: branch, commit, tag, etc.
--name:argo-source
-path:/src
-git:
-repo:https://github.com/argoproj/argo-workflows.git
-revision:"main"
-# Download kubectl 1.8.0 and place it at /bin/kubectl
--name:kubectl
-path:/bin/kubectl
-mode:0755
-http:
-url:https://storage.googleapis.com/kubernetes-release/release/v1.8.0/bin/linux/amd64/kubectl
-# Copy an s3 compatible artifact repository bucket (such as AWS, GCS and MinIO) and place it at /s3
--name:objects
-path:/s3
-s3:
-endpoint:storage.googleapis.com
-bucket:my-bucket-name
-key:path/in/bucket
-accessKeySecret:
-name:my-s3-credentials
-key:accessKey
-secretKeySecret:
-name:my-s3-credentials
-key:secretKey
-container:
-image:debian
-command:[sh,-c]
-args:["ls-l/src/bin/kubectl/s3"]
-
Below, we run the same container on a Kubernetes cluster using an Argo workflow template. Be sure to read the comments
-as they provide useful explanations.
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow# new type of k8s spec
-metadata:
-generateName:hello-world-# name of the workflow spec
-spec:
-entrypoint:whalesay# invoke the whalesay template
-templates:
--name:whalesay# name of the template
-container:
-image:docker/whalesay
-command:[cowsay]
-args:["helloworld"]
-resources:# limit the resources
-limits:
-memory:32Mi
-cpu:100m
-
-
Argo adds a new kind of Kubernetes spec called a Workflow. The above spec contains a single template
-called whalesay which runs the docker/whalesay container and invokes cowsay "hello world". The whalesay template
-is the entrypoint for the spec. The entrypoint specifies the initial template that should be invoked when the workflow
-spec is executed by Kubernetes. Being able to specify the entrypoint is more useful when there is more than one template
-defined in the Kubernetes workflow spec. :-)
Argo is implemented as a Kubernetes CRD (Custom Resource Definition). As a result, Argo workflows can be managed
-using kubectl and natively integrates with other Kubernetes services such as volumes, secrets, and RBAC. The new Argo
-software is light-weight and installs in under a minute, and provides complete workflow features including parameter
-substitution, artifacts, fixtures, loops and recursive workflows.
In many cases, you will want to manage Kubernetes resources from Argo workflows. The resource template allows you to create, delete or updated any type of Kubernetes resource.
-
# in a workflow. The resource template type accepts any k8s manifest
-# (including CRDs) and can perform any `kubectl` action against it (e.g. create,
-# apply, delete, patch).
-apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:k8s-jobs-
-spec:
-entrypoint:pi-tmpl
-templates:
--name:pi-tmpl
-resource:# indicates that this is a resource template
-action:create# can be any kubectl action (e.g. create, delete, apply, patch)
-# The successCondition and failureCondition are optional expressions.
-# If failureCondition is true, the step is considered failed.
-# If successCondition is true, the step is considered successful.
-# They use kubernetes label selection syntax and can be applied against any field
-# of the resource (not just labels). Multiple AND conditions can be represented by comma
-# delimited expressions.
-# For more details: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
-successCondition:status.succeeded > 0
-failureCondition:status.failed > 3
-manifest:|#put your kubernetes spec here
-apiVersion: batch/v1
-kind: Job
-metadata:
-generateName: pi-job-
-spec:
-template:
-metadata:
-name: pi
-spec:
-containers:
-- name: pi
-image: perl
-command: ["perl", "-Mbignum=bpi", "-wle", "print bpi(2000)"]
-restartPolicy: Never
-backoffLimit: 4
-
-
Note:
-Currently only a single resource can be managed by a resource template so either a generateName or name must be provided in the resource's meta-data.
-
Resources created in this way are independent of the workflow. If you want the resource to be deleted when the workflow is deleted then you can use Kubernetes garbage collection with the workflow resource as an owner reference (example).
-
You can also collect data about the resource in output parameters (see more at k8s-jobs.yaml)
-
Note:
-When patching, the resource will accept another attribute, mergeStrategy, which can either be strategic, merge, or json. If this attribute is not supplied, it will default to strategic. Keep in mind that Custom Resources cannot be patched with strategic, so a different strategy must be chosen. For example, suppose you have the CronTab CRD defined, and the following instance of a CronTab:
When writing workflows, it is often very useful to be able to iterate over a set of inputs, as this is how argo-workflows can perform loops.
-
There are two basic ways of running a template multiple times.
-
-
withItems takes a list of things to work on. Either
-
plain, single values, which are then usable in your template as '{{item}}'
-
a JSON object where each element in the object can be addressed by it's key as '{{item.key}}'
-
-
-
withParam takes a JSON array of items, and iterates over it - again the items can be objects like with withItems. This is very powerful, as you can generate the JSON in another step in your workflow, so creating a dynamic workflow.
This example is the simplest. We are taking a basic list of items and iterating over it with withItems. It is limited to one varying field for each of the workflow templates instantiated.
If we'd like to pass more than one piece of information in each workflow, you can instead use a JSON object for each entry in withItems and then address the elements by key, as shown in this example.
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:loops-maps-
-spec:
-entrypoint:loop-map-example
-templates:
--name:loop-map-example# parameter specifies the list to iterate over
-steps:
---name:test-linux
-template:cat-os-release
-arguments:
-parameters:
--name:image
-value:"{{item.image}}"
--name:tag
-value:"{{item.tag}}"
-withItems:
--{ image:'debian', tag:'9.1'}#item set 1
--{ image:'debian', tag:'8.9'}#item set 2
--{ image:'alpine', tag:'3.6'}#item set 3
--{ image:'ubuntu', tag:'17.10'}#item set 4
-
--name:cat-os-release
-inputs:
-parameters:
--name:image
--name:tag
-container:
-image:"{{inputs.parameters.image}}:{{inputs.parameters.tag}}"
-command:[cat]
-args:[/etc/os-release]
-
This example does exactly the same job as the previous example, but using withParam to pass the information as a JSON array argument, instead of hard-coding it into the template.
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:loops-param-arg-
-spec:
-entrypoint:loop-param-arg-example
-arguments:
-parameters:
--name:os-list# a list of items
-value:|
-[
-{ "image": "debian", "tag": "9.1" },
-{ "image": "debian", "tag": "8.9" },
-{ "image": "alpine", "tag": "3.6" },
-{ "image": "ubuntu", "tag": "17.10" }
-]
-
-templates:
--name:loop-param-arg-example
-inputs:
-parameters:
--name:os-list
-steps:
---name:test-linux
-template:cat-os-release
-arguments:
-parameters:
--name:image
-value:"{{item.image}}"
--name:tag
-value:"{{item.tag}}"
-withParam:"{{inputs.parameters.os-list}}"# parameter specifies the list to iterate over
-
-# This template is the same as in the previous example
--name:cat-os-release
-inputs:
-parameters:
--name:image
--name:tag
-container:
-image:"{{inputs.parameters.image}}:{{inputs.parameters.tag}}"
-command:[cat]
-args:[/etc/os-release]
-
-
withParam example from another step in the workflow¶
-
Finally, the most powerful form of this is to generate that JSON array of objects dynamically in one step, and then pass it to the next step so that the number and values used in the second step are only calculated at runtime.
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:loops-param-result-
-spec:
-entrypoint:loop-param-result-example
-templates:
--name:loop-param-result-example
-steps:
---name:generate
-template:gen-number-list
-# Iterate over the list of numbers generated by the generate step above
---name:sleep
-template:sleep-n-sec
-arguments:
-parameters:
--name:seconds
-value:"{{item}}"
-withParam:"{{steps.generate.outputs.result}}"
-
-# Generate a list of numbers in JSON format
--name:gen-number-list
-script:
-image:python:alpine3.6
-command:[python]
-source:|
-import json
-import sys
-json.dump([i for i in range(20, 31)], sys.stdout)
-
--name:sleep-n-sec
-inputs:
-parameters:
--name:seconds
-container:
-image:alpine:latest
-command:[sh,-c]
-args:["echosleepingfor{{inputs.parameters.seconds}}seconds;sleep{{inputs.parameters.seconds}};echodone"]
-
The output of all iterations can be accessed as a JSON array, once the loop is done.
-The example below shows how you can read it.
-
Please note: the output of each iteration must be a valid JSON.
-
apiVersion:argoproj.io/v1alpha1
-kind:WorkflowTemplate
-metadata:
-name:loop-test
-spec:
-entrypoint:main
-templates:
--name:main
-steps:
---name:execute-parallel-steps
-template:print-json-entry
-arguments:
-parameters:
--name:index
-value:'{{item}}'
-withParam:'[1,2,3]'
---name:call-access-aggregate-output
-template:access-aggregate-output
-arguments:
-parameters:
--name:aggregate-results
-# If the value of each loop iteration isn't a valid JSON,
-# you get a JSON parse error:
-value:'{{steps.execute-parallel-steps.outputs.result}}'
--name:print-json-entry
-inputs:
-parameters:
--name:index
-# The output must be a valid JSON
-script:
-image:alpine:latest
-command:[sh]
-source:|
-cat <<EOF
-{
-"input": "{{inputs.parameters.index}}",
-"transformed-input": "{{inputs.parameters.index}}.jpeg"
-}
-EOF
--name:access-aggregate-output
-inputs:
-parameters:
--name:aggregate-results
-value:'no-value'
-script:
-image:alpine:latest
-command:[sh]
-source:|
-echo 'inputs.parameters.aggregate-results: "{{inputs.parameters.aggregate-results}}"'
-
-
-
The last step of the workflow above should have this output:
-inputs.parameters.aggregate-results: "[{"input":"1","transformed-input":"1.jpeg"},{"input":"2","transformed-input":"2.jpeg"},{"input":"3","transformed-input":"3.jpeg"}]"
Output parameters provide a general mechanism to use the result of a step as a parameter (and not just as an artifact). This allows you to use the result from any type of step, not just a script, for conditional tests, loops, and arguments. Output parameters work similarly to script result except that the value of the output parameter is set to the contents of a generated file rather than the contents of stdout.
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:output-parameter-
-spec:
-entrypoint:output-parameter
-templates:
--name:output-parameter
-steps:
---name:generate-parameter
-template:whalesay
---name:consume-parameter
-template:print-message
-arguments:
-parameters:
-# Pass the hello-param output from the generate-parameter step as the message input to print-message
--name:message
-value:"{{steps.generate-parameter.outputs.parameters.hello-param}}"
-
--name:whalesay
-container:
-image:docker/whalesay:latest
-command:[sh,-c]
-args:["echo-nhelloworld>/tmp/hello_world.txt"]# generate the content of hello_world.txt
-outputs:
-parameters:
--name:hello-param# name of output parameter
-valueFrom:
-path:/tmp/hello_world.txt# set the value of hello-param to the contents of this hello-world.txt
-
--name:print-message
-inputs:
-parameters:
--name:message
-container:
-image:docker/whalesay:latest
-command:[cowsay]
-args:["{{inputs.parameters.message}}"]
-
-
DAG templates use the tasks prefix to refer to another task, for example {{tasks.generate-parameter.outputs.parameters.hello-param}}.
The result output parameter captures standard output.
-It is accessible from the outputs map: outputs.result.
-Only 256 kb of the standard output stream will be captured.
Container steps and tasks also have their standard output captured in the result parameter.
-Given a task, called log-int, result would then be accessible as {{ tasks.log-int.outputs.result }}. If using steps, substitute tasks for steps: {{ steps.log-int.outputs.result }}.
Let's look at a slightly more complex workflow spec with parameters.
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:hello-world-parameters-
-spec:
-# invoke the whalesay template with
-# "hello world" as the argument
-# to the message parameter
-entrypoint:whalesay
-arguments:
-parameters:
--name:message
-value:hello world
-
-templates:
--name:whalesay
-inputs:
-parameters:
--name:message# parameter declaration
-container:
-# run cowsay with that message input parameter as args
-image:docker/whalesay
-command:[cowsay]
-args:["{{inputs.parameters.message}}"]
-
-
This time, the whalesay template takes an input parameter named message that is passed as the args to the cowsay command. In order to reference parameters (e.g., "{{inputs.parameters.message}}"), the parameters must be enclosed in double quotes to escape the curly braces in YAML.
-
The argo CLI provides a convenient way to override parameters used to invoke the entrypoint. For example, the following command would bind the message parameter to "goodbye world" instead of the default "hello world".
In case of multiple parameters that can be overridden, the argo CLI provides a command to load parameters files in YAML or JSON format. Here is an example of that kind of parameter file:
Command-line parameters can also be used to override the default entrypoint and invoke any template in the workflow spec. For example, if you add a new version of the whalesay template called whalesay-caps but you don't want to change the default entrypoint, you can invoke this from the command line as follows:
By using a combination of the --entrypoint and -p parameters, you can call any template in the workflow spec with any parameter that you like.
-
The values set in the spec.arguments.parameters are globally scoped and can be accessed via {{workflow.parameters.parameter_name}}. This can be useful to pass information to multiple steps in a workflow. For example, if you wanted to run your workflows with different logging levels that are set in the environment of each container, you could have a YAML file similar to this one:
In this workflow, both steps A and B would have the same log-level set to INFO and can easily be changed between workflow submissions using the -p flag.
In the first run, the coin immediately comes up heads and we stop. In the second run, the coin comes up tail three times before it finally comes up heads and we stop.
You can specify a retryStrategy that will dictate how failed or errored steps are retried:
-
# This example demonstrates the use of retry back offs
-apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:retry-backoff-
-spec:
-entrypoint:retry-backoff
-templates:
--name:retry-backoff
-retryStrategy:
-limit:10
-retryPolicy:"Always"
-backoff:
-duration:"1"# Must be a string. Default unit is seconds. Could also be a Duration, e.g.: "2m", "6h", "1d"
-factor:2
-maxDuration:"1m"# Must be a string. Default unit is seconds. Could also be a Duration, e.g.: "2m", "6h", "1d"
-affinity:
-nodeAntiAffinity:{}
-container:
-image:python:alpine3.6
-command:["python",-c]
-# fail with a 66% probability
-args:["importrandom;importsys;exit_code=random.choice([0,1,1]);sys.exit(exit_code)"]
-
-
-
limit is the maximum number of times the container will be retried.
-
retryPolicy specifies if a container will be retried on failure, error, both, or only transient errors (e.g. i/o or TLS handshake timeout). "Always" retries on both errors and failures. Also available: OnFailure (default), "OnError", and "OnTransientError" (available after v3.0.0-rc2).
-
backoff is an exponential back-off
-
nodeAntiAffinity prevents running steps on the same host. Current implementation allows only empty nodeAntiAffinity (i.e. nodeAntiAffinity: {}) and by default it uses label kubernetes.io/hostname as the selector.
-
-
Providing an empty retryStrategy (i.e. retryStrategy: {}) will cause a container to retry until completion.
Often, we just want a template that executes a script specified as a here-script (also known as a here document) in the workflow spec. This example shows how to do that:
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:scripts-bash-
-spec:
-entrypoint:bash-script-example
-templates:
--name:bash-script-example
-steps:
---name:generate
-template:gen-random-int-bash
---name:print
-template:print-message
-arguments:
-parameters:
--name:message
-value:"{{steps.generate.outputs.result}}"# The result of the here-script
-
--name:gen-random-int-bash
-script:
-image:debian:9.4
-command:[bash]
-source:|# Contents of the here-script
-cat /dev/urandom | od -N2 -An -i | awk -v f=1 -v r=100 '{printf "%i\n", f + r * $1 / 65536}'
-
--name:gen-random-int-python
-script:
-image:python:alpine3.6
-command:[python]
-source:|
-import random
-i = random.randint(1, 100)
-print(i)
-
--name:gen-random-int-javascript
-script:
-image:node:9.1-alpine
-command:[node]
-source:|
-var rand = Math.floor(Math.random() * 100);
-console.log(rand);
-
--name:print-message
-inputs:
-parameters:
--name:message
-container:
-image:alpine:latest
-command:[sh,-c]
-args:["echoresultwas:{{inputs.parameters.message}}"]
-
-
The script keyword allows the specification of the script body using the source tag. This creates a temporary file containing the script body and then passes the name of the temporary file as the final parameter to command, which should be an interpreter that executes the script body.
-
The use of the script feature also assigns the standard output of running the script to a special output parameter named result. This allows you to use the result of running the script itself in the rest of the workflow spec. In this example, the result is simply echoed by the print-message template.
Argo supports the same secrets syntax and mechanisms as Kubernetes Pod specs, which allows access to secrets as environment variables or volume mounts. See the Kubernetes documentation for more information.
-
# To run this example, first create the secret by running:
-# kubectl create secret generic my-secret --from-literal=mypassword=S00perS3cretPa55word
-apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:secret-example-
-spec:
-entrypoint:whalesay
-# To access secrets as files, add a volume entry in spec.volumes[] and
-# then in the container template spec, add a mount using volumeMounts.
-volumes:
--name:my-secret-vol
-secret:
-secretName:my-secret# name of an existing k8s secret
-templates:
--name:whalesay
-container:
-image:alpine:3.7
-command:[sh,-c]
-args:['
-echo"secretfromenv:$MYSECRETPASSWORD";
-echo"secretfromfile:`cat/secret/mountpath/mypassword`"
-']
-# To access secrets as environment variables, use the k8s valueFrom and
-# secretKeyRef constructs.
-env:
--name:MYSECRETPASSWORD# name of env var
-valueFrom:
-secretKeyRef:
-name:my-secret# name of an existing k8s secret
-key:mypassword# 'key' subcomponent of the secret
-volumeMounts:
--name:my-secret-vol# mount file containing secret at /secret/mountpath
-mountPath:"/secret/mountpath"
-
A sidecar is another container that executes concurrently in the same pod as the main container and is useful in creating multi-container pods.
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:sidecar-nginx-
-spec:
-entrypoint:sidecar-nginx-example
-templates:
--name:sidecar-nginx-example
-container:
-image:appropriate/curl
-command:[sh,-c]
-# Try to read from nginx web server until it comes up
-args:["until`curl-G'http://127.0.0.1/'>&/tmp/out`;doechosleep&&sleep1;done&&cat/tmp/out"]
-# Create a simple nginx web server
-sidecars:
--name:nginx
-image:nginx:1.13
-command:[nginx,-g,daemon off;]
-
-
In the above example, we create a sidecar container that runs Nginx as a simple web server. The order in which containers come up is random, so in this example the main container polls the Nginx container until it is ready to service requests. This is a good design pattern when designing multi-container systems: always wait for any services you need to come up before running your main code.
In this example, we'll see how to create multi-step workflows, how to define more than one template in a workflow spec, and how to create nested workflows. Be sure to read the comments as they provide useful explanations.
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:steps-
-spec:
-entrypoint:hello-hello-hello
-
-# This spec contains two templates: hello-hello-hello and whalesay
-templates:
--name:hello-hello-hello
-# Instead of just running a container
-# This template has a sequence of steps
-steps:
---name:hello1# hello1 is run before the following steps
-template:whalesay
-arguments:
-parameters:
--name:message
-value:"hello1"
---name:hello2a# double dash => run after previous step
-template:whalesay
-arguments:
-parameters:
--name:message
-value:"hello2a"
--name:hello2b# single dash => run in parallel with previous step
-template:whalesay
-arguments:
-parameters:
--name:message
-value:"hello2b"
-
-# This is the same template as from the previous example
--name:whalesay
-inputs:
-parameters:
--name:message
-container:
-image:docker/whalesay
-command:[cowsay]
-args:["{{inputs.parameters.message}}"]
-
-
The above workflow spec prints three different flavors of "hello". The hello-hello-hello template consists of three steps. The first step named hello1 will be run in sequence whereas the next two steps named hello2a and hello2b will be run in parallel with each other. Using the argo CLI command, we can graphically display the execution history of this workflow spec, which shows that the steps named hello2a and hello2b ran in parallel with each other.
We now know enough about the basic components of a workflow spec. To review its basic structure:
-
-
Kubernetes header including meta-data
-
-
Spec body
-
-
Entrypoint invocation with optional arguments
-
List of template definitions
-
-
-
-
For each template definition
-
-
Name of the template
-
Optionally a list of inputs
-
Optionally a list of outputs
-
Container invocation (leaf template) or a list of steps
-
For each step, a template invocation
-
-
-
-
-
-
To summarize, workflow specs are composed of a set of Argo templates where each template consists of an optional input section, an optional output section and either a container invocation or a list of steps where each step invokes another template.
-
Note that the container section of the workflow spec will accept the same options as the container section of a pod spec, including but not limited to environment variables, secrets, and volume mounts. Similarly, for volume claims and volumes.
The following example dynamically creates a volume and then uses the volume in a two step workflow.
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:volumes-pvc-
-spec:
-entrypoint:volumes-pvc-example
-volumeClaimTemplates:# define volume, same syntax as k8s Pod spec
--metadata:
-name:workdir# name of volume claim
-spec:
-accessModes:["ReadWriteOnce"]
-resources:
-requests:
-storage:1Gi# Gi => 1024 * 1024 * 1024
-
-templates:
--name:volumes-pvc-example
-steps:
---name:generate
-template:whalesay
---name:print
-template:print-message
-
--name:whalesay
-container:
-image:docker/whalesay:latest
-command:[sh,-c]
-args:["echogeneratingmessageinvolume;cowsayhelloworld|tee/mnt/vol/hello_world.txt"]
-# Mount workdir volume at /mnt/vol before invoking docker/whalesay
-volumeMounts:# same syntax as k8s Pod spec
--name:workdir
-mountPath:/mnt/vol
-
--name:print-message
-container:
-image:alpine:latest
-command:[sh,-c]
-args:["echogettingmessagefromvolume;find/mnt/vol;cat/mnt/vol/hello_world.txt"]
-# Mount workdir volume at /mnt/vol before invoking docker/whalesay
-volumeMounts:# same syntax as k8s Pod spec
--name:workdir
-mountPath:/mnt/vol
-
-
Volumes are a very useful way to move large amounts of data from one step in a workflow to another. Depending on the system, some volumes may be accessible concurrently from multiple steps.
-
In some cases, you want to access an already existing volume rather than creating/destroying one dynamically.
-
# Define Kubernetes PVC
-kind:PersistentVolumeClaim
-apiVersion:v1
-metadata:
-name:my-existing-volume
-spec:
-accessModes:["ReadWriteOnce"]
-resources:
-requests:
-storage:1Gi
-
----
-apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:volumes-existing-
-spec:
-entrypoint:volumes-existing-example
-volumes:
-# Pass my-existing-volume as an argument to the volumes-existing-example template
-# Same syntax as k8s Pod spec
--name:workdir
-persistentVolumeClaim:
-claimName:my-existing-volume
-
-templates:
--name:volumes-existing-example
-steps:
---name:generate
-template:whalesay
---name:print
-template:print-message
-
--name:whalesay
-container:
-image:docker/whalesay:latest
-command:[sh,-c]
-args:["echogeneratingmessageinvolume;cowsayhelloworld|tee/mnt/vol/hello_world.txt"]
-volumeMounts:
--name:workdir
-mountPath:/mnt/vol
-
--name:print-message
-container:
-image:alpine:latest
-command:[sh,-c]
-args:["echogettingmessagefromvolume;find/mnt/vol;cat/mnt/vol/hello_world.txt"]
-volumeMounts:
--name:workdir
-mountPath:/mnt/vol
-
-
It's also possible to declare existing volumes at the template level, instead of the workflow level.
-Workflows can generate volumes using a resource step.
Many clients can send events via the events API endpoint using a standard authorization header. However, for clients that are unable to do so (e.g. because they use signature verification as proof of origin), additional configuration is required.
-
In the namespace that will receive the event, create access token resources for your client:
-
-
A role with permissions to get workflow templates and to create a workflow: example
Widgets are intended to be embedded into other applications using inline frames (iframe). This may not work with your configuration. You may need to:
-
-
Run the Argo Server with an account that can read workflows. That can be done using --auth-mode=server and configuring the argo-server service account.
-
Run the Argo Server with --x-frame-options=SAMEORIGIN or --x-frame-options=.
The Argo server and the workflow controller currently only run on Linux. The workflow executor however also runs on Windows nodes, meaning you can use Windows containers inside your workflows! Here are the steps to get started.
If you're running workflows in your hybrid Kubernetes cluster, always make sure to include a nodeSelector to run the steps on the correct host OS:
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:hello-windows-
-spec:
-entrypoint:hello-win
-templates:
--name:hello-win
-nodeSelector:
-kubernetes.io/os:windows# specify the OS your step should run on
-container:
-image:mcr.microsoft.com/windows/nanoserver:1809
-command:["cmd","/c"]
-args:["echo","HellofromWindowsContainer!"]
-
-
You can run this example and get the logs:
-
$argosubmit--watchhttps://raw.githubusercontent.com/argoproj/argo-workflows/main/examples/hello-windows.yaml
-$argologshello-windows-s9kk5
-hello-windows-s9kk5:"Hello from Windows Container!"
-
You can also run different steps on different host operating systems. This can for example be very helpful when you need to compile your application on Windows and Linux.
$argosubmit--watchhttps://raw.githubusercontent.com/argoproj/argo-workflows/main/examples/hello-hybrid.yaml
-$argologshello-hybrid-plqpp
-hello-hybrid-plqpp-1977432187:"Hello from Windows Container!"
-hello-hybrid-plqpp-764774907:HellofromLinuxContainer!
-
Artifacts work mostly the same way as on Linux. All paths get automatically mapped to the C: drive. For example:
-
# ...
--name:print-message
-inputs:
-artifacts:
-# unpack the message input artifact
-# and put it at C:\message
--name:message
-path:"/message"# gets mapped to C:\message
-nodeSelector:
-kubernetes.io/os:windows
-container:
-image:mcr.microsoft.com/windows/nanoserver:1809
-command:["cmd","/c"]
-args:["dirC:\\message"]# List the C:\message directory
-
Sharing process namespaces doesn't work on Windows so you can't use the Process Namespace Sharing (PNS) workflow executor.
-
The executor Windows container is built using Nano Server as the base image. Running a newer windows version (e.g. 1909) is currently not confirmed to be working. If this is required, you need to build the executor container yourself by first adjusting the base image.
To build the workflow executor image for Windows you need a Windows machine running Windows Server 2019 with Docker installed like described in the docs.
-
You then clone the project and run the Docker build with the Dockerfile for Windows and argoexec as a target:
You can make workflows faster and more robust by employing work avoidance. A workflow that utilizes this is simply a workflow containing steps that do not run if the work has already been done.
-
This is a technique is similar to memoization. Work avoidance is totally in your control and you make the decisions as to have to skip the work. Memoization is a feature of Argo Workflows to automatically skip steps which generate outputs. Prior to version 3.5 this required outputs to be specified, but you can use memoization for all steps and tasks in version 3.5 or later.
-
This simplest way to do this is to use marker files.
-
Use cases:
-
-
An expensive step appears across multiple workflows - you want to avoid repeating them.
-
A workflow has unreliable tasks - you want to be able to resubmit the workflow.
-
-
A marker file is a file that indicates the work has already been done. Before doing the work you check to see if the marker has already been done:
If you want to keep completed workflows for a long time, you can use the workflow archive to save them in a Postgres or MySQL (>= 5.7.8) database.
-The workflow archive stores the status of the workflow, which pods have been executed, what was the result etc.
-The job logs of the workflow pods will not be archived.
-If you need to save the logs of the pods, you must setup an artifact repository according to this doc.
-
The quick-start deployment includes a Postgres database server.
-In this case the workflow archive is already enabled.
-Such a deployment is convenient for test environments, but in a production environment you must use a production quality database service.
To enable archiving of the workflows, you must configure database parameters in the persistence section of your configuration and set archive: to true.
Note that IAM-based authentication is not currently supported. However, you can start your database proxy as a sidecar
-(e.g. via CloudSQL Proxy on GCP) and then specify your local
-proxy address, IAM username, and an empty string as your password in the persistence configuration to connect to it.
-
The following tables will be created in the database when you start the workflow controller with enabled archive:
Every time the Argo workflow-controller starts with persistence enabled, it tries to migrate the database to the correct version.
-If the database migration fails, the workflow-controller will also fail to start.
-In this case you can delete all the above tables and restart the workflow-controller.
-
If you know what are you doing you also have an option to skip migration:
The database user/role must have CREATE and USAGE permissions on the public schema of the database so that the tables can be created during the migration.
You can configure the time period to keep archived workflows before they will be deleted by the archived workflow garbage collection function.
-The default is forever.
-
Example:
-
persistence:
-archiveTTL:10d
-
-
-
The ARCHIVED_WORKFLOW_GC_PERIOD variable defines the periodicity of running the garbage collection function.
-The default value is documented here.
-When the workflow controller starts, it sets the ticker to run every ARCHIVED_WORKFLOW_GC_PERIOD.
-It does not run the garbage collection function immediately and the first garbage collection happens only after the period defined in the ARCHIVED_WORKFLOW_GC_PERIOD variable.
The Workflow is the most important resource in Argo and serves two important functions:
-
-
It defines the workflow to be executed.
-
It stores the state of the workflow.
-
-
Because of these dual responsibilities, a Workflow should be treated as a "live" object. It is not only a static definition, but is also an "instance" of said definition. (If it isn't clear what this means, it will be explained below).
The workflow to be executed is defined in the Workflow.spec field. The core structure of a Workflow spec is a list of templates and an entrypoint.
-
templates can be loosely thought of as "functions": they define instructions to be executed.
-The entrypoint field defines what the "main" function will be – that is, the template that will be executed first.
-
Here is an example of a simple Workflow spec with a single template:
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:hello-world-# Name of this Workflow
-spec:
-entrypoint:whalesay# Defines "whalesay" as the "main" template
-templates:
--name:whalesay# Defining the "whalesay" template
-container:
-image:docker/whalesay
-command:[cowsay]
-args:["helloworld"]# This template runs "cowsay" in the "whalesay" image with arguments "hello world"
-
Perhaps the most common template type, it will schedule a Container. The spec of the template is the same as the Kubernetes container spec, so you can define a container here the same way you do anywhere else in Kubernetes.
A convenience wrapper around a container. The spec is the same as a container, but adds the source: field which allows you to define a script in-place.
-The script will be saved into a file and executed for you. The result of the script is automatically exported into an Argo variable either {{tasks.<NAME>.outputs.result}} or {{steps.<NAME>.outputs.result}}, depending how it was called.
A suspend template will suspend execution, either for a duration or until it is resumed manually. Suspend templates can be resumed from the CLI (with argo resume), the API endpoint, or the UI.
A steps template allows you to define your tasks in a series of steps. The structure of the template is a "list of lists". Outer lists will run sequentially and inner lists will run in parallel. If you want to run inner lists one by one, use the Synchronization feature. You can set a wide array of options to control execution, such as when: clauses to conditionally execute a step.
-
In this example step1 runs first. Once it is completed, step2a and step2b will run in parallel:
A dag template allows you to define your tasks as a graph of dependencies. In a DAG, you list all your tasks and set which other tasks must complete before a particular task can begin. Tasks without any dependencies will be run immediately.
-
In this example A runs first. Once it is completed, B and C will run in parallel and once they both complete, D will run:
In all versions, the configuration may be under a config: | key:
-
# This file describes the config settings available in the workflow controller configmap
-apiVersion:v1
-kind:ConfigMap
-metadata:
-name:workflow-controller-configmap
-data:
-config:|
-instanceID: my-ci-controller
-artifactRepository:
-archiveLogs: true
-s3:
-endpoint: s3.amazonaws.com
-bucket: my-bucket
-region: us-west-2
-insecure: false
-accessKeySecret:
-name: my-s3-credentials
-key: accessKey
-secretKeySecret:
-name: my-s3-credentials
-key: secretKey
-
-
In version 2.7+, the config: | key is optional. However, if the config: | key is not used, all nested maps under top level
-keys should be strings. This makes it easier to generate the map with some configuration management tools like Kustomize.
-
# This file describes the config settings available in the workflow controller configmap
-apiVersion:v1
-kind:ConfigMap
-metadata:
-name:workflow-controller-configmap
-data:# "config: |" key is optional in 2.7+!
-instanceID:my-ci-controller
-artifactRepository:|# However, all nested maps must be strings
-archiveLogs: true
-s3:
-endpoint: s3.amazonaws.com
-bucket: my-bucket
-region: us-west-2
-insecure: false
-accessKeySecret:
-name: my-s3-credentials
-key: accessKey
-secretKeySecret:
-name: my-s3-credentials
-key: secretKey
-
If you create your workflow via the CLI or UI, an attempt will be made to label it with the user who created it
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-name:my-wf
-labels:
-workflows.argoproj.io/creator:admin
-# labels must be DNS formatted, so the "@" is replaces by '.at.'
-workflows.argoproj.io/creator-email:admin.at.your.org
-workflows.argoproj.io/creator-preferred-username:admin-preferred-username
-
-
-
Note
-
Labels only contain [-_.0-9a-zA-Z], so any other characters will be turned into -.
⚠️ Do not use Kubernetes events for automation. Events maybe lost or rolled-up.
-
We emit Kubernetes events on certain events.
-
Workflow state change:
-
-
WorkflowRunning
-
WorkflowSucceeded
-
WorkflowFailed
-
WorkflowTimedOut
-
-
Node state change:
-
-
WorkflowNodeRunning
-
WorkflowNodeSucceeded
-
WorkflowNodeFailed
-
WorkflowNodeError
-
-
The involved object is the workflow in both cases. Additionally, for node state change events, annotations indicate the name and type of the involved node:
A workflow executor is a process that conforms to a specific interface that allows Argo to perform certain actions like monitoring pod logs, collecting artifacts, managing container life-cycles, etc.
-
The executor to be used in your workflows can be changed in the config map under the containerRuntimeExecutor key (removed in v3.4).
If you don't provide command to run, the emissary will grab it from container image. You can also specify it using the workflow spec or emissary will look it up in the image index. This is nothing more fancy than
-a configuration item.
-
Emissary will create a cache entry, using image with version as key and command as value, and it will reuse it for specific image/version.
It requires privileged access to docker.sock of the host to be mounted which. Often rejected by Open Policy Agent (OPA) or your Pod Security Policy (PSP).
-
It can escape the privileges of the pod's service account
Output artifacts must be saved on volumes (e.g. empty-dir) and not the base image layer (e.g. /tmp)
-
-
-
Step/Task result:
-
Warnings that normally goes to stderr will get captured in a step or a dag task's outputs.result. May require changes if your pipeline is conditioned on steps/tasks.name.outputs.result
Log retrieval and container operations performed against the remote Kubernetes API
-
-
-
Artifacts:
-
Output artifacts must be saved on volumes (e.g. empty-dir) and not the base image layer (e.g. /tmp)
-
-
-
Step/Task result:
-
Warnings that normally goes to stderr will get captured in a step or a dag task's outputs.result. May require changes if your pipeline is conditioned on steps/tasks.name.outputs.result
Processes are visible to other containers in the pod. This includes all information visible in /proc, such as passwords that were passed as arguments or environment variables. These are protected only by regular Unix permissions.
-
-
-
Scalable:
-
Most operations use local procfs.
-
Log retrieval uses the remote Kubernetes API
-
-
-
Artifacts:
-
Output artifacts can be located on the base layer (e.g. /tmp)
-
Cannot capture artifacts from a base layer which has a volume mounted under it
-
Cannot capture artifacts from base layer if the container is short-lived.
Workflows and templates operate on a set of defined parameters and arguments that are supplied to the running container. The precise details of how to manage the inputs can be confusing; this article attempts to clarify concepts and provide simple working examples to illustrate the various configuration options.
-
The examples below are limited to DAGTemplates and mainly focused on parameters, but similar reasoning applies to the other types of templates.
First, some clarification of terms is needed. For a glossary reference, see Argo Core Concepts.
-
A workflow provides arguments, which are passed in to the entry point template. A template defines inputs which are then provided by template callers (such as steps, dag, or even a workflow). The structure of both is identical.
-
For example, in a Workflow, one parameter would look like this:
In DAGTemplates, it is common to want to take the output of one step and send it as the input to another step. However, there is a difference in how this works for artifacts vs parameters. Suppose our step-template-a defines some outputs:
All pods in a workflow run with the service account specified in workflow.spec.serviceAccountName, or if omitted,
-the default service account of the workflow's namespace. The amount of access which a workflow needs is dependent on
-what the workflow needs to do. For example, if your workflow needs to deploy a resource, then the workflow's service
-account will require 'create' privileges on that resource.
-
Warning: We do not recommend using the default service account in production. It is a shared account so may have
-permissions added to it you do not want. Instead, create a service account only for your workflow.
As the administrator of the controller, you may want to limit which types of Workflows your users can run.
-Workflow Restrictions allow you to set requirements for all Workflows.
templateReferencing: Strict: Only process Workflows using workflowTemplateRef. You can use this to require usage of WorkflowTemplates, disallowing arbitrary Workflow execution.
-
templateReferencing: Secure: Same as Strictplus enforce that a referenced WorkflowTemplate hasn't changed between operations. If a running Workflow's underlying WorkflowTemplate changes, the Workflow will error out.
For example, to specify that Workflows may only run with workflowTemplateRef:
-
# This file describes the config settings available in the workflow controller configmap
-apiVersion:v1
-kind:ConfigMap
-metadata:
-name:workflow-controller-configmap
-data:
-workflowRestrictions:|
-templateReferencing: Strict
-
If you want one workflow to create another, you can do this using curl. You'll need an access token. Typically the best way is to submit from a workflow template:
WorkflowTemplates are definitions of Workflows that live in your cluster. This allows you to create a library of
-frequently-used templates and reuse them either by submitting them directly (v2.7 and after) or by referencing them from
-your Workflows.
The terms WorkflowTemplate and template have created an unfortunate naming collision and have created some confusion
-in the past. However, a quick description should clarify each and their differences.
-
-
A template (lower-case) is a task within a Workflow or (confusingly) a WorkflowTemplate under the field templates. Whenever you define a
-Workflow, you must define at least one (but usually more than one) template to run. This template can be of type
-container, script, dag, steps, resource, or suspend and can be referenced by an entrypoint or by other
-dag, and step templates.
-
-
Here is an example of a Workflow with two templates:
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:steps-
-spec:
-entrypoint:hello# We reference our first "template" here
-
-templates:
--name:hello# The first "template" in this Workflow, it is referenced by "entrypoint"
-steps:# The type of this "template" is "steps"
---name:hello
-template:whalesay# We reference our second "template" here
-arguments:
-parameters:[{name:message, value:"hello1"}]
-
--name:whalesay# The second "template" in this Workflow, it is referenced by "hello"
-inputs:
-parameters:
--name:message
-container:# The type of this "template" is "container"
-image:docker/whalesay
-command:[cowsay]
-args:["{{inputs.parameters.message}}"]
-
-
-
A WorkflowTemplate is a definition of a Workflow that lives in your cluster. Since it is a definition of a Workflow
-it also contains templates. These templates can be referenced from within the WorkflowTemplate and from other Workflows
-and WorkflowTemplates on your cluster. To see how, please see Referencing Other WorkflowTemplates.
In v2.7 and after, all the fields in WorkflowSpec (except for priority that must be configured in a WorkflowSpec itself) are supported for WorkflowTemplates. You can take any existing Workflow you may have and convert it to a WorkflowTemplate by substituting kind: Workflow to kind: WorkflowTemplate.
-
-
v2.4 – 2.6
-
-
WorkflowTemplates in v2.4 - v2.6 are only partial Workflow definitions and only support the templates and
-arguments field.
-
This would not be a valid WorkflowTemplate in v2.4 - v2.6 (notice entrypoint field):
-
apiVersion:argoproj.io/v1alpha1
-kind:WorkflowTemplate
-metadata:
-name:workflow-template-submittable
-spec:
-entrypoint:whalesay-template# Fields other than "arguments" and "templates" not supported in v2.4 - v2.6
-arguments:
-parameters:
--name:message
-value:hello world
-templates:
--name:whalesay-template
-inputs:
-parameters:
--name:message
-container:
-image:docker/whalesay
-command:[cowsay]
-args:["{{inputs.parameters.message}}"]
-
When working with parameters in a WorkflowTemplate, please note the following:
-
-
When working with global parameters, you can instantiate your global variables in your Workflow
-and then directly reference them in your WorkflowTemplate. Below is a working example:
When working with local parameters, the values of local parameters must be supplied at the template definition inside
-the WorkflowTemplate. Below is a working example:
You can reference templates from another WorkflowTemplates (see the difference between the two) using a templateRef field.
-Just as how you reference other templates within the same Workflow, you should do so from a steps or dag template.
-
Here is an example from a steps template:
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:workflow-template-hello-world-
-spec:
-entrypoint:whalesay
-templates:
--name:whalesay
-steps:# You should only reference external "templates" in a "steps" or "dag" "template".
---name:call-whalesay-template
-templateRef:# You can reference a "template" from another "WorkflowTemplate" using this field
-name:workflow-template-1# This is the name of the "WorkflowTemplate" CRD that contains the "template" you want
-template:whalesay-template# This is the name of the "template" you want to reference
-arguments:# You can pass in arguments as normal
-parameters:
--name:message
-value:"helloworld"
-
You should never reference another template directly on a template object (outside of a steps or dag template).
-This includes both using template and templateRef.
-This behavior is deprecated, no longer supported, and will be removed in a future version.
-
Here is an example of a deprecated reference that should not be used:
-
apiVersion:argoproj.io/v1alpha1
-kind:Workflow
-metadata:
-generateName:workflow-template-hello-world-
-spec:
-entrypoint:whalesay
-templates:
--name:whalesay
-template:# You should NEVER use "template" here. Use it under a "steps" or "dag" template (see above).
-templateRef:# You should NEVER use "templateRef" here. Use it under a "steps" or "dag" template (see above).
-name:workflow-template-1
-template:whalesay-template
-arguments:# Arguments here are ignored. Use them under a "steps" or "dag" template (see above).
-parameters:
--name:message
-value:"helloworld"
-
-
The reasoning for deprecating this behavior is that a template is a "definition": it defines inputs and things to be
-done once instantiated. With this deprecated behavior, the same template object is allowed to be an "instantiator":
-to pass in "live" arguments and reference other templates (those other templates may be "definitions" or "instantiators").
-
This behavior has been problematic and dangerous. It causes confusion and has design inconsistencies.
You can create Workflow from WorkflowTemplate spec using workflowTemplateRef. If you pass the arguments to created Workflow, it will be merged with workflow template arguments.
-Here is an example for referring WorkflowTemplate as Workflow with passing entrypoint and Workflow Arguments to WorkflowTemplate
-
Open the playlist