Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expand buildpack author concepts pages #739

Merged
44 changes: 42 additions & 2 deletions content/docs/for-buildpack-authors/concepts/build-plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,48 @@ title="What is the build plan?"
weight=99
+++

A **build plan** is a `toml` file that is the output of the [detect](https://buildpacks.io/docs/for-buildpack-authors/concepts/lifecycle-phases/#phase-2-detect) phase, in which each component buildpack or image extension may express the dependencies it requires and the dependencies it provides.

hyounes4560 marked this conversation as resolved.
Show resolved Hide resolved
<!--more-->

This page is a stub! The CNB project is applying to [Google Season of Docs](https://developers.google.com/season-of-docs/docs/timeline) to receive support for improving our documentation. Please check back soon.
## Example Build Plan (toml)

In order to make contributions to the `Build Plan`, a `/bin/detect` executable MUST write entries to `<plan>` in two sections: `requires` and `provides`. The generated `plan.toml` file is usually added under the `<layers>`directory.
hyounes4560 marked this conversation as resolved.
Show resolved Hide resolved
The `requires` and `provides` sections MAY be repeated together inside of an `or` array at the top-level.
Each `requires` and `provides` section MUST be a list of entries formatted as shown below:

```toml
[[provides]]
name = "<dependency name>"

[[requires]]
name = "<dependency name>"

[requires.metadata]
# buildpack-specific data

[[or]]

[[or.provides]]
name = "<dependency name>"

[[or.requires]]
name = "<dependency name>"

[or.requires.metadata]
# buildpack-specific data

```

## Key Points

* A valid `build plan` is a plan where all required dependencies are provided in the necessary order, meaning that during the `build` phase, each component buildpack will have its required dependencies provided by component buildpack or an image extension that runs before it.
* Each pairing of `requires` and `provides` sections (at the top level, or inside of an `or` array) is a potential build plan. For more details, see the [JVM buildpack](https://buildpacks.io/docs/for-buildpack-authors/how-to/write-buildpacks/use-build-plan/#example-jvm-buildpack) example.
* A group will only pass detection if a valid build plan can be produced from the dependencies that all elements in the group require and provide.
* The `detect` phase could fail if a buildpack requires a dependency that it does not itself provide, or is not provided by another buildpack.
* The `detect` phase could also fail when the buildpacks order is incorrect, i.e, the buildpacks providing dependencies run `after` the buildpacks requiring them.
* The resulting `build plan` is passed as one of the inputs to the `build` phase.

## Resources

If you are familiar with this content and would like to make a contribution, please feel free to open a PR :)
For more about build plans, look at [Use the build plan]( https://buildpacks.io/docs/for-buildpack-authors/how-to/write-buildpacks/use-build-plan/)
hyounes4560 marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

+++
title="What is a buildpack group?"
aliases=[
Expand All @@ -7,7 +6,7 @@ aliases=[
weight=2
+++

A buildpack group is a list of individual buildpacks that are designged to work together to build an application.
A buildpack group is a list of individual buildpacks that are designed to work together to build an application.

hyounes4560 marked this conversation as resolved.
Show resolved Hide resolved
<!--more-->

Expand Down Expand Up @@ -58,4 +57,4 @@ The [Operator's Guide][operator-guide] has more information on creating builders
[order-resolution]: https://github.com/buildpacks/spec/blob/main/buildpack.md#order-resolution
[operator-guide]: /docs/for-platform-operators/
[builder]: /docs/for-platform-operators/concepts/builder/
[composite buildpack]: /docs/for-platform-operators/concepts/composite-buildpack
[composite buildpack]: /docs/for-buildpack-authors/concepts/composite-buildpack
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

+++
title="What caching strategies are available to buildpacks?"
aliases=[
Expand All @@ -8,7 +7,7 @@ weight=4
summary="Learn strategies for caching layers at build-time for future re-use."
+++

# Layers
## Layers

There are three types of layers that can be contributed to an image

Expand All @@ -18,7 +17,7 @@ There are three types of layers that can be contributed to an image

In this section we look at caching each layer type.

## Layer Metadata
### Layer Metadata

buildpacks ensure byte-for-byte reproducibility of layers. File creation time is [normalized to January 1, 1980](https://medium.com/buildpacks/time-travel-with-pack-e0efd8bf05db) to ensure reproducibility. Byte-for-byte reproducibility means previous layers can be reused. However, we may want to invalidate previously cached layers if an important property changes, such as:

Expand All @@ -27,7 +26,7 @@ buildpacks ensure byte-for-byte reproducibility of layers. File creation time i

Launch layers are exported to an OCI registry. The layer metadata is commonly used when deciding if a launch layer should be re-used. A launch layer may be re-used on an OCI registry without downloading the layer to the machine running a build.

## Caching Strategies
### Caching Strategies

Caching during the production of an application image is necessarily very flexible. Most buildpacks that wish to contribute a layer to the application image need only to

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,22 @@ title="What is a component buildpack?"
weight=99
+++

A **component buildpack** is a buildpack containing `/bin/detect` and `/bin/build` executables and that implements the [Buildpack Interface](https://github.com/buildpacks/spec/blob/main/buildpack.md#buildpack-interface).

<!--more-->

This page is a stub! The CNB project is applying to [Google Season of Docs](https://developers.google.com/season-of-docs/docs/timeline) to receive support for improving our documentation. Please check back soon.
## Key Points

During the `build` phase, typical component buildpacks might perform one, or more, of the following actions:

If you are familiar with this content and would like to make a contribution, please feel free to open a PR :)
* Read the [Buildpack Plan](https://buildpacks.io/docs/for-buildpack-authors/concepts/build-plan/) in `<plan>` to determine what dependencies to provide
* Supply the application with [dependencies](https://buildpacks.io/docs/for-buildpack-authors/concepts/layer/) for launch in `<layers>/<layer>`
* Reuse application [dependencies](https://buildpacks.io/docs/for-buildpack-authors/how-to/write-buildpacks/re-use-layers/) from a previous image by appending `[types]` and `launch = true` to `<layers>/<layer>.toml`
* Contribute [dependencies](https://buildpacks.io/docs/for-buildpack-authors/concepts/layer/) added in `<layers>/<layer>` to subsequent buildpacks
* Reuse [cached build dependencies](https://buildpacks.io/docs/for-buildpack-authors/how-to/write-buildpacks/create-layer/) from a previous build by appending `[types]`, `build = true` and `cache = true` to `<layers>/<layer>.toml`
* Compile the application source code into object code
* Remove application source code that is not necessary for launch
* Supply start command in `<layers>/launch.toml`
* Write a partial [Software Bill of Materials](https://buildpacks.io/docs/for-buildpack-authors/how-to/write-buildpacks/add-sbom/) to `<layers>/<layer>.sbom.<ext>` describing any dependencies provided in the layer
* Write a partial [Software Bill of Materials](https://buildpacks.io/docs/for-buildpack-authors/how-to/write-buildpacks/add-sbom/) to `<layers>/launch.sbom.<ext>` describing any provided application dependencies not associated with a layer
* Write a partial [Software Bill of Materials](https://buildpacks.io/docs/for-buildpack-authors/how-to/write-buildpacks/add-sbom/) to `<layers>/build.sbom.<ext>` describing any provided build dependencies not associated with a layer
24 changes: 22 additions & 2 deletions content/docs/for-buildpack-authors/concepts/layer.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,28 @@ title="What is a buildpack dependency layer?"
weight=99
+++

`Dependency layers` are semantically meaningful layers that are contributed by one or more buildpacks during the `build` phase, one for each dependency.

<!--more-->

This page is a stub! The CNB project is applying to [Google Season of Docs](https://developers.google.com/season-of-docs/docs/timeline) to receive support for improving our documentation. Please check back soon.
![builder](/images/builder.svg)

As seen in the image above, buildpacks read application source code and create dependency layers. Each buildpack can contribute a subset of an app's required dependencies, added as subdirectories under the `CNB_LAYERS_DIR` directory. These dependencies are then exported as layers in the final app image or build cache.

The `build` phase runs the `/bin/build` binary of each buildpack, which outputs zero or more layers into `$(CNB_LAYERS_DIR)/<buildpack-id>` and writes metadata for each layer as `toml` files in that directory. During the `export` phase, all layers created by the buildpacks are either cached or added to the output application image.

The following shows an example filesystem tree created after the `build` phase is complete:

```text

layers/
├── buildpack-1-id
│   ├── layer-1
│   ├── layer-1.toml
│   ├── layer-2
│   └── layer-2.toml
└── buildpack-2-id

```

If you are familiar with this content and would like to make a contribution, please feel free to open a PR :)
> For more information about creating `dependency layers`, see [Create dependency layers](https://buildpacks.io/docs/for-buildpack-authors/how-to/write-buildpacks/create-layer/)
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

+++
title="What is the lifecycle?"
aliases=[
Expand Down
6 changes: 4 additions & 2 deletions content/docs/for-buildpack-authors/concepts/package.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,10 @@ title="What is a buildpack package?"
weight=99
+++

A `buildpack package` refers to the process of packaging buildpacks for distribution as OCI images or OCI-compatible `tar` files.

<!--more-->

This page is a stub! The CNB project is applying to [Google Season of Docs](https://developers.google.com/season-of-docs/docs/timeline) to receive support for improving our documentation. Please check back soon.
A `buildpack package` typically encapsulates the scripts, required dependencies, and metadata needed to transform application source code into a fully operational application within a specific platform's runtime environment.

If you are familiar with this content and would like to make a contribution, please feel free to open a PR :)
For more information about packaging a buildpack, see [Package a buildpack or extension](https://buildpacks.io/docs/for-buildpack-authors/how-to/distribute-buildpacks/package-buildpack/).
3 changes: 2 additions & 1 deletion content/docs/for-buildpack-authors/concepts/targets.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

+++
title="What are targets?"
aliases=[
Expand All @@ -12,6 +11,7 @@ The concept of `targets` is used to identify compatibility between buildpacks an
<!--more-->

Target data includes:

* Operating system name (e.g., "linux")
* Architecture (e.g., "amd64", "arm64")
* Architecture variant
Expand All @@ -25,6 +25,7 @@ Buildpacks may declare the targets they are compatible with in `buildpack.toml`.
This information will be used by `pack` (or other platforms) and the lifecycle to avoid running buildpacks on images they aren't designed to work with.

Additionally, during builds this information will be read by the lifecycle from the run image and exposed to buildpacks through `CNB_TARGET_` environment variables:

* `CNB_TARGET_OS`
* `CNB_TARGET_ARCH`
* `CNB_TARGET_ARCH_VARIANT`
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title="Use the build plan"
weight=2
+++

The [Build Plan](https://github.com/buildpacks/spec/blob/main/buildpack.md#build-plan-toml) is a document that buildpacks can use to pass information between the `detect` and `build` phases, and between each other.
The [Build Plan](https://buildpacks.io/docs/for-buildpack-authors/concepts/build-plan/) is a document that buildpacks can use to pass information between the `detect` and `build` phases, and between each other.
The build plan is passed (by the lifecycle) as a parameter to the `detect` and `build` binaries of each buildpack.

<!--more-->
Expand All @@ -24,25 +24,28 @@ Let's see how this works with an example.

Let's walk through some possible cases a `node-engine` buildpack may consider:

1. Nothing in the app explicitly calls out that it is needed
2. It is explicitly referred to in some configuration file
1. Nothing in the app explicitly calls out that it is needed
2. It is explicitly referred to in some configuration file

We will also consider what an `NPM` and a `JVM` buildpack may do.

#### Scenario 1: No Explicit Request

A `node-engine` buildpack is always happy to `provide` the `node` dependency. The build plan it will write may look something like:
```

```toml
[[provides]]
name = "node"
```

> **NOTE:** If this was the only buildpack running, this would fail the `detect` phase. In order to pass, every `provides` must be matched up with a `requires`, whether in the same buildpack or in another buildpack.
> See the [spec](https://github.com/buildpacks/spec/blob/main/buildpack.md#phase-1-detection) for particulars on how ordering buildpacks can adjust detection results.

#### Scenario 2: One Version Requested

During the `detect` phase, the `node-engine` buildpack sees in one configuration file (e.g. a `.nvmrc` file in the app directory) that `node v10.x` is explicitly requested by the application. Seeing that, it may write the below text to the build plan:
```

```toml
[[provides]]
name = "node"

Expand All @@ -63,13 +66,15 @@ As always, the buildpack `provides` `node`. In this particular case, a version o
NPM is typically distributed together with node. As a result, a NPM buildpack may require `node`, but not want to `provide` it, trusting that the `node-engine` buildpack will be in charge of `providing` `node`.

The NPM buildpack could write the following to the build plan, if the buildpack sees that `npm` is necessary (e.g., it sees a `package.json` file in the app directory):
```

```toml
[[requires]]
name = "node"
```

If, looking in the `package.json` file, the NPM buildpack sees a specific version of `node` requested in the [engines](https://docs.npmjs.com/files/package.json#engines) field (e.g. `14.1`), it may write the following to the build plan:
```

```toml
[[requires]]
name = "node"
version = "14.1"
Expand All @@ -90,7 +95,7 @@ A very naive implementation of the buildpack may have it write several `provides
while later buildpacks would figure out based on the application which options it requires, and would `require` those.
In this particular case, we can use the `or` operator to present different possible build plans the buildpack can follow:

```
```toml
# option 1 (`jre` and `jdk`)
[[provides]]
name = "jre"
Expand All @@ -110,6 +115,7 @@ name = "jre"
```

The buildpack gives three options to the lifecycle:

* It can provide a standalone `jre`
* It can provide a standalone `jdk`
* It can provide both the `jdk` and `jre`
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ The [buildpacks `exec.d` interface](https://github.com/buildpacks/spec/blob/main
* **Inputs**
* A third open file descriptor. File descriptors are integers used by a process to uniquely identify opened files; pre-opened file descriptors are usually `0` for `stdin`, `1` for `stdout` and `2` for `stderr`. The third open file descriptor is inherited from the calling process.
* **Outputs**
* Valid TOML describing environment variables in the form of key=value pairs. These variables are added to the application's runtime environment. The content should be written to file descriptor 3 (see examples for how to do this).
* Valid TOML describing environment variables in the form of `key="value"` pairs. These variables are added to the application's runtime environment. The content should be written to file descriptor 3 (see examples for how to do this).
* Exit Code: The scripts should exit with a status code of `0` to indicate success. A non-zero exit code will indicate an error and prevent the application from launching.

## Use Cases
Expand Down
Loading