Skip to content

Latest commit

 

History

History
986 lines (831 loc) · 22 KB

buildx_bake.md

File metadata and controls

986 lines (831 loc) · 22 KB

buildx bake

docker buildx bake [OPTIONS] [TARGET...]

Build from a file

Aliases

bake, f

Options

Name Type Default Description
--builder string Override the configured builder instance
-f, --file stringArray Build definition file
--load Shorthand for --set=*.output=type=docker
--metadata-file string Write build result metadata to the file
--no-cache Do not use cache when building the image
--print Print the options without building
--progress string auto Set type of progress output (auto, plain, tty). Use plain to show container output
--pull Always attempt to pull all referenced images
--push Shorthand for --set=*.output=type=registry
--set stringArray Override target value (e.g., targetpattern.key=value)

Description

Bake is a high-level build command. Each specified target will run in parallel as part of the build.

Read High-level build options for introduction.

Please note that buildx bake command may receive backwards incompatible features in the future if needed. We are looking for feedback on improving the command and extending the functionality further.

Examples

Override the configured builder instance (--builder)

Same as buildx --builder.

Specify a build definition file (-f, --file)

By default, buildx bake looks for build definition files in the current directory, the following are parsed:

  • docker-compose.yml
  • docker-compose.yaml
  • docker-bake.json
  • docker-bake.override.json
  • docker-bake.hcl
  • docker-bake.override.hcl

Use the -f / --file option to specify the build definition file to use. The file can be a Docker Compose, JSON or HCL file. If multiple files are specified they are all read and configurations are combined.

The following example uses a Docker Compose file named docker-compose.dev.yaml as build definition file, and builds all targets in the file:

$ docker buildx bake -f docker-compose.dev.yaml

[+] Building 66.3s (30/30) FINISHED
 => [frontend internal] load build definition from Dockerfile  0.1s
 => => transferring dockerfile: 36B                            0.0s
 => [backend internal] load build definition from Dockerfile   0.2s
 => => transferring dockerfile: 3.73kB                         0.0s
 => [database internal] load build definition from Dockerfile  0.1s
 => => transferring dockerfile: 5.77kB                         0.0s
 ...

Pass the names of the targets to build, to build only specific target(s). The following example builds the backend and database targets that are defined in the docker-compose.dev.yaml file, skipping the build for the frontend target:

$ docker buildx bake -f docker-compose.dev.yaml backend database

[+] Building 2.4s (13/13) FINISHED
 => [backend internal] load build definition from Dockerfile  0.1s
 => => transferring dockerfile: 81B                           0.0s
 => [database internal] load build definition from Dockerfile 0.2s
 => => transferring dockerfile: 36B                           0.0s
 => [backend internal] load .dockerignore                     0.3s
 ...

You can also use a remote git bake definition:

$ docker buildx bake "https://github.com/docker/cli.git#v20.10.11" --print
#1 [internal] load git source https://github.com/docker/cli.git#v20.10.11
#1 0.745 e8f1871b077b64bcb4a13334b7146492773769f7       refs/tags/v20.10.11
#1 2.022 From https://github.com/docker/cli
#1 2.022  * [new tag]         v20.10.11  -> v20.10.11
#1 DONE 2.9s
{
  "group": {
    "default": {
      "targets": [
        "binary"
      ]
    }
  },
  "target": {
    "binary": {
      "context": "https://github.com/docker/cli.git#v20.10.11",
      "dockerfile": "Dockerfile",
      "args": {
        "BASE_VARIANT": "alpine",
        "GO_STRIP": "",
        "VERSION": ""
      },
      "target": "binary",
      "platforms": [
        "local"
      ],
      "output": [
        "build"
      ]
    }
  }
}

As you can see the context is fixed to https://github.com/docker/cli.git even if no context is actually defined in the definition.

If you want to access the main context for bake command from a bake file that has been imported remotely, you can use the BAKE_CMD_CONTEXT builtin var:

$ cat https://raw.githubusercontent.com/tonistiigi/buildx/remote-test/docker-bake.hcl
target "default" {
  context = BAKE_CMD_CONTEXT
  dockerfile-inline = <<EOT
FROM alpine
WORKDIR /src
COPY . .
RUN ls -l && stop
EOT
}
$ docker buildx bake "https://github.com/tonistiigi/buildx.git#remote-test" --print
{
  "target": {
    "default": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "dockerfile-inline": "FROM alpine\nWORKDIR /src\nCOPY . .\nRUN ls -l \u0026\u0026 stop\n"
    }
  }
}
$ touch foo bar
$ docker buildx bake "https://github.com/tonistiigi/buildx.git#remote-test"
...
 > [4/4] RUN ls -l && stop:
#8 0.101 total 0
#8 0.102 -rw-r--r--    1 root     root             0 Jul 27 18:47 bar
#8 0.102 -rw-r--r--    1 root     root             0 Jul 27 18:47 foo
#8 0.102 /bin/sh: stop: not found
$ docker buildx bake "https://github.com/tonistiigi/buildx.git#remote-test" "https://github.com/docker/cli.git#v20.10.11" --print
#1 [internal] load git source https://github.com/tonistiigi/buildx.git#remote-test
#1 0.429 577303add004dd7efeb13434d69ea030d35f7888       refs/heads/remote-test
#1 CACHED
{
  "target": {
    "default": {
      "context": "https://github.com/docker/cli.git#v20.10.11",
      "dockerfile": "Dockerfile",
      "dockerfile-inline": "FROM alpine\nWORKDIR /src\nCOPY . .\nRUN ls -l \u0026\u0026 stop\n"
    }
  }
}
$ docker buildx bake "https://github.com/tonistiigi/buildx.git#remote-test" "https://github.com/docker/cli.git#v20.10.11"
...
 > [4/4] RUN ls -l && stop:
#8 0.136 drwxrwxrwx    5 root     root          4096 Jul 27 18:31 kubernetes
#8 0.136 drwxrwxrwx    3 root     root          4096 Jul 27 18:31 man
#8 0.136 drwxrwxrwx    2 root     root          4096 Jul 27 18:31 opts
#8 0.136 -rw-rw-rw-    1 root     root          1893 Jul 27 18:31 poule.yml
#8 0.136 drwxrwxrwx    7 root     root          4096 Jul 27 18:31 scripts
#8 0.136 drwxrwxrwx    3 root     root          4096 Jul 27 18:31 service
#8 0.136 drwxrwxrwx    2 root     root          4096 Jul 27 18:31 templates
#8 0.136 drwxrwxrwx   10 root     root          4096 Jul 27 18:31 vendor
#8 0.136 -rwxrwxrwx    1 root     root          9620 Jul 27 18:31 vendor.conf
#8 0.136 /bin/sh: stop: not found

Do not use cache when building the image (--no-cache)

Same as build --no-cache. Do not use cache when building the image.

Print the options without building (--print)

Prints the resulting options of the targets desired to be built, in a JSON format, without starting a build.

$ docker buildx bake -f docker-bake.hcl --print db
{
  "group": {
    "default": {
      "targets": [
        "db"
      ]
    }
  },
  "target": {
    "db": {
      "context": "./",
      "dockerfile": "Dockerfile",
      "tags": [
        "docker.io/tiborvass/db"
      ]
    }
  }
}

Set type of progress output (--progress)

Same as build --progress. Set type of progress output (auto, plain, tty). Use plain to show container output (default "auto").

You can also use the BUILDKIT_PROGRESS environment variable to set its value.

The following example uses plain output during the build:

$ docker buildx bake --progress=plain

#2 [backend internal] load build definition from Dockerfile.test
#2 sha256:de70cb0bb6ed8044f7b9b1b53b67f624e2ccfb93d96bb48b70c1fba562489618
#2 ...

#1 [database internal] load build definition from Dockerfile.test
#1 sha256:453cb50abd941762900a1212657a35fc4aad107f5d180b0ee9d93d6b74481bce
#1 transferring dockerfile: 36B done
#1 DONE 0.1s
...

Always attempt to pull a newer version of the image (--pull)

Same as build --pull.

Override target configurations from command line (--set)

--set targetpattern.key[.subkey]=value

Override target configurations from command line. The pattern matching syntax is defined in https://golang.org/pkg/path/#Match.

$ docker buildx bake --set target.args.mybuildarg=value
$ docker buildx bake --set target.platform=linux/arm64
$ docker buildx bake --set foo*.args.mybuildarg=value # overrides build arg for all targets starting with 'foo'
$ docker buildx bake --set *.platform=linux/arm64     # overrides platform for all targets
$ docker buildx bake --set foo*.no-cache              # bypass caching only for targets starting with 'foo'

Complete list of overridable fields: args, cache-from, cache-to, context, dockerfile, labels, no-cache, output, platform, pull, secrets, ssh, tags, target

File definition

In addition to compose files, bake supports a JSON and an equivalent HCL file format for defining build groups and targets.

A target reflects a single docker build invocation with the same options that you would specify for docker build. A group is a grouping of targets.

Multiple files can include the same target and final build options will be determined by merging them together.

In the case of compose files, each service corresponds to a target.

A group can specify its list of targets with the targets option. A target can inherit build options by setting the inherits option to the list of targets or groups to inherit from.

Note: Design of bake command is work in progress, the user experience may change based on feedback.

HCL definition example:

group "default" {
    targets = ["db", "webapp-dev"]
}

target "webapp-dev" {
    dockerfile = "Dockerfile.webapp"
    tags = ["docker.io/username/webapp"]
}

target "webapp-release" {
    inherits = ["webapp-dev"]
    platforms = ["linux/amd64", "linux/arm64"]
}

target "db" {
    dockerfile = "Dockerfile.db"
    tags = ["docker.io/username/db"]
}

Complete list of valid target fields:

args, cache-from, cache-to, context, contexts, dockerfile, inherits, labels, no-cache, no-cache-filter, output, platform, pull, secrets, ssh, tags, target

Global scope attributes

You can define global scope attributes in HCL/JSON and use them for code reuse and setting values for variables. This means you can do a "data-only" HCL file with the values you want to set/override and use it in the list of regular output files.

# docker-bake.hcl
variable "FOO" {
    default = "abc"
}

target "app" {
    args = {
        v1 = "pre-${FOO}"
    }
}

You can use this file directly:

$ docker buildx bake --print app
{
  "group": {
    "default": {
      "targets": [
        "app"
      ]
    }
  },
  "target": {
    "app": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "args": {
        "v1": "pre-abc"
      }
    }
  }
}

Or create an override configuration file:

# env.hcl
WHOAMI="myuser"
FOO="def-${WHOAMI}"

And invoke bake together with both of the files:

$ docker buildx bake -f docker-bake.hcl -f env.hcl --print app
{
  "group": {
    "default": {
      "targets": [
        "app"
      ]
    }
  },
  "target": {
    "app": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "args": {
        "v1": "pre-def-myuser"
      }
    }
  }
}

HCL variables and functions

Similar to how Terraform provides a way to define variables, the HCL file format also supports variable block definitions. These can be used to define variables with values provided by the current environment, or a default value when unset.

A set of generally useful functions provided by go-cty are available for use in HCL files. In addition, user defined functions are also supported.

Using interpolation to tag an image with the git sha

Bake supports variable blocks which are assigned to matching environment variables or default values.

# docker-bake.hcl
variable "TAG" {
    default = "latest"
}

group "default" {
    targets = ["webapp"]
}

target "webapp" {
    tags = ["docker.io/username/webapp:${TAG}"]
}

alternatively, in json format:

{
  "variable": {
    "TAG": {
      "default": "latest"
    }
  }
  "group": {
    "default": {
      "targets": ["webapp"]
    }
  },
  "target": {
    "webapp": {
      "tags": ["docker.io/username/webapp:${TAG}"]
    }
  }
}
$ docker buildx bake --print webapp
{
  "group": {
    "default": {
      "targets": [
        "webapp"
      ]
    }
  },
  "target": {
    "webapp": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "tags": [
        "docker.io/username/webapp:latest"
      ]
    }
  }
}
$ TAG=$(git rev-parse --short HEAD) docker buildx bake --print webapp
{
  "group": {
    "default": {
      "targets": [
        "webapp"
      ]
    }
  },
  "target": {
    "webapp": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "tags": [
        "docker.io/username/webapp:985e9e9"
      ]
    }
  }
}

Using the add function

You can use go-cty stdlib functions. Here we are using the add function.

# docker-bake.hcl
variable "TAG" {
    default = "latest"
}

group "default" {
    targets = ["webapp"]
}

target "webapp" {
    args = {
        buildno = "${add(123, 1)}"
    }
}
$ docker buildx bake --print webapp
{
  "group": {
    "default": {
      "targets": [
        "webapp"
      ]
    }
  },
  "target": {
    "webapp": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "args": {
        "buildno": "124"
      }
    }
  }
}

Defining an increment function

It also supports user defined functions. The following example defines a simple an increment function.

# docker-bake.hcl
function "increment" {
    params = [number]
    result = number + 1
}

group "default" {
    targets = ["webapp"]
}

target "webapp" {
    args = {
        buildno = "${increment(123)}"
    }
}
$ docker buildx bake --print webapp
{
  "group": {
    "default": {
      "targets": [
        "webapp"
      ]
    }
  },
  "target": {
    "webapp": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "args": {
        "buildno": "124"
      }
    }
  }
}

Only adding tags if a variable is not empty using an notequal

Here we are using the conditional notequal function which is just for symmetry with the equal one.

# docker-bake.hcl
variable "TAG" {default="" }

group "default" {
    targets = [
        "webapp",
    ]
}

target "webapp" {
    context="."
    dockerfile="Dockerfile"
    tags = [
        "my-image:latest",
        notequal("",TAG) ? "my-image:${TAG}": "",
    ]
}
$ docker buildx bake --print webapp
{
  "group": {
    "default": {
      "targets": [
        "webapp"
      ]
    }
  },
  "target": {
    "webapp": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "tags": [
        "my-image:latest"
      ]
    }
  }
}

Using variables in functions

You can refer variables to other variables like the target blocks can. Stdlib functions can also be called but user functions can't at the moment.

# docker-bake.hcl
variable "REPO" {
    default = "user/repo"
}

function "tag" {
    params = [tag]
    result = ["${REPO}:${tag}"]
}

target "webapp" {
    tags = tag("v1")
}
$ docker buildx bake --print webapp
{
  "group": {
    "default": {
      "targets": [
        "webapp"
      ]
    }
  },
  "target": {
    "webapp": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "tags": [
        "user/repo:v1"
      ]
    }
  }
}

Using variables in variables across files

When multiple files are specified, one file can use variables defined in another file.

# docker-bake1.hcl
variable "FOO" {
    default = upper("${BASE}def")
}

variable "BAR" {
    default = "-${FOO}-"
}

target "app" {
    args = {
        v1 = "pre-${BAR}"
    }
}
# docker-bake2.hcl
variable "BASE" {
    default = "abc"
}

target "app" {
    args = {
        v2 = "${FOO}-post"
    }
}
$ docker buildx bake -f docker-bake1.hcl -f docker-bake2.hcl --print app
{
  "group": {
    "default": {
      "targets": [
        "app"
      ]
    }
  },
  "target": {
    "app": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "args": {
        "v1": "pre--ABCDEF-",
        "v2": "ABCDEF-post"
      }
    }
  }
}

Using typed variables

Non-string variables are also accepted. The value passed with env is parsed into suitable type first.

# docker-bake.hcl
variable "FOO" {
    default = 3
}

variable "IS_FOO" {
    default = true
}

target "app" {
    args = {
        v1 = FOO > 5 ? "higher" : "lower" 
        v2 = IS_FOO ? "yes" : "no"
    }
}
$ docker buildx bake --print app
{
  "group": {
    "default": {
      "targets": [
        "app"
      ]
    }
  },
  "target": {
    "app": {
      "context": ".",
      "dockerfile": "Dockerfile",
      "args": {
        "v1": "lower",
        "v2": "yes"
      }
    }
  }
}

Defining additional build contexts and linking targets

In addition to the main context key that defines the build context each target can also define additional named contexts with a map defined with key contexts. These values map to the --build-context flag in the build command.

Inside the Dockerfile these contexts can be used with the FROM instruction or --from flag.

The value can be a local source directory, container image (with docker-image:// prefix), Git URL, HTTP URL or a name of another target in the Bake file (with target: prefix).

Pinning alpine image

# Dockerfile
FROM alpine
RUN echo "Hello world"
# docker-bake.hcl
target "app" {
    contexts = {
        alpine = "docker-image://alpine:3.13"
    }
}

Using a secondary source directory

# Dockerfile

FROM scratch AS src

FROM golang
COPY --from=src . .
# docker-bake.hcl
target "app" {
    contexts = {
        src = "../path/to/source"
    }
}

Using a result of one target as a base image in another target

To use a result of one target as a build context of another, specity the target name with target: prefix.

# Dockerfile
FROM baseapp
RUN echo "Hello world"
# docker-bake.hcl

target "base" {
    dockerfile = "baseapp.Dockerfile"
}

target "app" {
    contexts = {
        baseapp = "target:base"
    }
}

Please note that in most cases you should just use a single multi-stage Dockerfile with multiple targets for similar behavior. This case is recommended when you have multiple Dockerfiles that can't be easily merged into one.

Extension field with Compose

Special extension field x-bake can be used in your compose file to evaluate fields that are not (yet) available in the build definition.

# docker-compose.yml
services:
  addon:
    image: ct-addon:bar
    build:
      context: .
      dockerfile: ./Dockerfile
      args:
        CT_ECR: foo
        CT_TAG: bar
      x-bake:
        tags:
          - ct-addon:foo
          - ct-addon:alp
        platforms:
          - linux/amd64
          - linux/arm64
        cache-from:
          - user/app:cache
          - type=local,src=path/to/cache
        cache-to: type=local,dest=path/to/cache
        pull: true

  aws:
    image: ct-fake-aws:bar
    build:
      dockerfile: ./aws.Dockerfile
      args:
        CT_ECR: foo
        CT_TAG: bar
      x-bake:
        secret:
          - id=mysecret,src=./secret
          - id=mysecret2,src=./secret2
        platforms: linux/arm64
        output: type=docker
        no-cache: true
$ docker buildx bake --print
{
  "group": {
    "default": {
      "targets": [
        "aws",
        "addon"
      ]
    }
  },
  "target": {
    "addon": {
      "context": ".",
      "dockerfile": "./Dockerfile",
      "args": {
        "CT_ECR": "foo",
        "CT_TAG": "bar"
      },
      "tags": [
        "ct-addon:foo",
        "ct-addon:alp"
      ],
      "cache-from": [
        "user/app:cache",
        "type=local,src=path/to/cache"
      ],
      "cache-to": [
        "type=local,dest=path/to/cache"
      ],
      "platforms": [
        "linux/amd64",
        "linux/arm64"
      ],
      "pull": true
    },
    "aws": {
      "context": ".",
      "dockerfile": "./aws.Dockerfile",
      "args": {
        "CT_ECR": "foo",
        "CT_TAG": "bar"
      },
      "tags": [
        "ct-fake-aws:bar"
      ],
      "secret": [
        "id=mysecret,src=./secret",
        "id=mysecret2,src=./secret2"
      ],
      "platforms": [
        "linux/arm64"
      ],
      "output": [
        "type=docker"
      ],
      "no-cache": true
    }
  }
}

Complete list of valid fields for x-bake:

tags, cache-from, cache-to, secret, ssh, platforms, output, pull, no-cache, no-cache-filter

Built-in variables

  • BAKE_CMD_CONTEXT can be used to access the main context for bake command from a bake file that has been imported remotely.
  • BAKE_LOCAL_PLATFORM returns the current platform's default platform specification (e.g. linux/amd64).