Steps

Steps

A pipeline is made up for steps, and a step defines a list of commands to run inside a docker container.

All steps are executed using the workspace as the working directory.

Here is an example step to build a maven project:

steps:
  - name: build-jar
    image: maven:3-jdk-8-alpine
    commands:
      - mvn clean package

Environment

In addition to the standard build environment variables, steps can be configured by adding a environment section to the step, with arbitrary step-specific configuration. These can also reference previously defined variables.

steps:
  - name: docker-tag
    image: ${plugins}/docker
    environment:
      IMAGE_TAG: ${PIPELINE_APP_VERSION}.${PIPELINE_BUILD_NUMBER}
    commands:
      - docker tag my_container:latest my_container:${IMAGE_TAG}

Complex data

In the environment section, we allow arbitrary YAML data. This means that in addition to passing simple scalar values, you can specify configuration as a list or object. Please refer the examples and explanation provided under Pipeline-Fundamentals.

Conditions

We support defining conditional pipeline steps in a when block. If all conditions in the when block evaluate to true, the step is executed, otherwise it is skipped.

pipeline:
  steps:
    - name: only-on-master
      image: alpine:3.8.1
      commands:
        - echo "I must be executing on a master branch build"
      when:
        branch: master

Status

By default, the pipeline aborts if a step fails. You can have a step run regardless of the pipeline status by adding:

pipeline:
  steps:
    - name: always-run
      image: alpine:3.8.1
      commands:
        - echo "Current status is $PIPELINE_STATUS, but I will always run"
      when:
        status: [ failure, success]

• status takes a list of statuses to execute on
• by default steps only execute on success
• available status are success, failure

Branch

Execute a step if the branch is on master or release:

when:
  branch: [master, release]

Execute a step if the branch starts with feature/*

when:
  branch: feature/*

Execute a step using include and exclude logic:

when:
  branch:
    include: [ master, feature/* ]
    exclude: [ feature/something, feature/old_thing* ]

Syntax

A branch condition can take a single value or pattern, a list of values or patterns, or a map with elements include and exclude, each of which can take a single value or list.

Patterns

We implement simple pattern matching, and can match against the following styles: xxx*, *xxx, *xxx*, and xxx*yyy, with an arbitrary number of pattern parts, as well as direct matches.

Environment

Control if a step executes based on environment variables.

The environment variables can be standard pipeline variables, job build variables, or user defined variables.

# skip this step if the commit message contains "skip ci"

when:
  environment:
    PIPELINE_COMMIT_MESSAGE:
      exclude: "*skip ci*"

# execute this step when deployment environment is in qlab

when:
  environment:
    DEPLOY_ENV: "qlab*"

Matches against multiple environment variables are supported. All matches must be successful to run the step.

when:
  environment:
    PIPELINE_COMMIT_AUTHOR: "larry*"
    PIPELINE_COMMIT_MESSAGE:
      exclude: [ "*skip ci*", "*skip build*" ]
    DEPLOY_ENV:
      include: [ "qlab*" ]
      exlcude: [ "qlab02", "qlab04" ]

Syntax

Like branch conditions, environment conditions can take a single value or pattern, a list of values or patterns, or a map with elements include and exclude, each of which can take a single value or list.

Patterns

We implement simple pattern matching, and can match against the following styles: xxx*, *xxx, *xxx*, and xxx*yyy, with an arbitrary number of pattern parts, as well as direct matches.

Secrets

Note: the secrets support is still in-flux as we work out integration with other systems

Jenkins Secrets

Currently you can provide credentials from Jenkins to a step as environment variables. We support the "Username with password" and "Secret text" credential types. Note that jenkins will do its best to mask the credentials from the console, but you should also avoid leaking credentials where possible.

Here's an of using "Username with password" credentials:

- name: publish-docs-container
    image: ${plugins}/docker:stable
    secrets:
    - source: docker_registry
      target:
        - DOCKER_USER
        - DOCKER_PASS
    commands:
    - echo ${DOCKER_PASS} | docker login -u ${DOCKER_USER} --password-stdin ${ARTIFACTORY}

As you can see, we specify the credential id under source. target takes a list of environment variables. For username with password, there should be two entries. For "secret text" there should be one entry.

Control-Options

timeoutInMinutes

By default, all job steps will timeout after 30 minutes (per step).

You can override for long running steps, by adding a timeoutInMinutes property:

- name: my-step
  image: ...
  timeoutInMinutes: 120 # step may run for 2 hours
  commands:
    - ...

Timeout Behavior

A timeout will be treated the same as any other step failure. That is, it will move the PIPELINE_STATE to FAILURE, and future jobs will not be executed unless they have conditions to run in FAILURE state. Additionally the overall job will be marked as FAILED.

The explanation field in the pipeline state will indicate a timeout occurred. E.g.

"stepStates": [
{
    "commands": [
       "gradle --no-daemon clean test jacocoTestReport"
    ],
    "durationMillis": 1860000,
    "image": "gradle:5.0-jre8-alpine",
    "startTime": 1554239761551,
    "name": "test-pipeline",
    "status": "FAILURE",
    "explanation": "30 minute timeout exceeded"
},

continueOnError

Normally if any step fails, the pipeline is marked FAILURE and future steps are not run (unless they explicitly elect to be run on failure using conditions).

It is possible to override this and essentially make a step's success optional, by adding the option continueOnError. By default, this is false. If set to true, the step will be marked as failed, but pipeline status will not be changed, and future steps will execute.

- name: my-optional-step-that-may-fail
  image: ...
  continueOnError: true
  commands:
    - ...